knight/datahub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[#809,docs][l]: fix Vercel build, add examples do sidebar, fix blogs list

Browse Source
This commit is contained in:
deme 2023-05-05 16:17:28 -03:00
parent 91caeff6c3
commit 0c65a145c8
5 changed files with 32 additions and 301 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
site/content/data-literate.md
View File

@ -1,21 +0,0 @@
---
title: Data Literate Documents
author: Rufus Pollock and Friends
---
**What?** An experiment in simple, lightweight approach to creating, displaying and sharing datasets and data-driven stories.
**Why?** a simple, fast, extensible way to present data(sets) and author data-driven content. I want to work with markdown for content and quickly add data in the simplest way possible e.g. dropping in links, pasting tables or adding links to the metadata.
**How?** Technically the essence is Markdown+React (MDX) + a curated toolkit of components for data-presentation + NextJS for framework and deployment.
Check out the [demo](/data-literate/demo).
## Background
I have observed two converging data-rich use cases:
* **Data Publishing**: quickly presenting data whether a single file or a full dataset.
* **Data Stories**: creating data-driven content from the simplest of a blog post with a graph to high end there is sophisticated data journalism and visualization.
Both of these can now be well served by a simple markdown-plus approach. Taking data publishing first. I've long been a fan of ultra-simple `README + metadata + csv` datasets. With the evolution of frontmatter we can merge the metadata into the README. However, we still need to "present" the dataset and the key thing for a dataset is the data and this is not something markdown ever supported well ... But now with MDX and the richness of the javascript ecosystem it's quite easy to enhance our markdown and build a rendering pipeleine.

268
site/content/data-literate/demo.mdx
View File

@ -1,268 +0,0 @@
---
title: Demo
---
This demos and documents Data Literate features live.
You can see the raw source of this page here: https://raw.githubusercontent.com/datopian/data-literate/main/content/demo.mdx
## Table of Contents
## GFM
We can have github-flavored markdown including markdown tables, auto-linked links and checklists:
```
https://github.com/datopian/portaljs
| a | b |
|---|---|
| 1 | 2 |
* [x] one thing to do
* [ ] a second thing to do
```
https://github.com/datopian/portaljs
| a | b |
|---|---|
| 1 | 2 |
* [x] one thing to do
* [ ] a second thing to do
## Footnotes
```
here is a footnote reference[^1]
[^1]: a very interesting footnote.
```
here is a footnote reference[^1]
[^1]: a very interesting footnote.
## Frontmatter
Posts can have frontmatter like:
```
---
title: Hello World
author: Rufus Pollock
---
```
The title and description are pulled from the MDX file and processed using `gray-matter`. Additionally, links are rendered using a custom component passed to `next-mdx-remote`.
## A Table of Contents
You can create a table of contents by having a markdown heading named `Table of Contents`. You can see an example at the start of this post.
## A Table
You can create tables ...
```
<Table cols={[
{ key: 'id', name: 'ID' },
{ key: 'firstName', name: 'First name' },
{ key: 'lastName', name: 'Last name' },
{ key: 'age', name: 'Age' }
]} data={[
{ id: 1, lastName: 'Snow', firstName: 'Jon', age: 35 },
{ id: 2, lastName: 'Lannister', firstName: 'Cersei', age: 42 },
{ id: 3, lastName: 'Lannister', firstName: 'Jaime', age: 45 },
{ id: 4, lastName: 'Stark', firstName: 'Arya', age: 16 },
{ id: 7, lastName: 'Clifford', firstName: 'Ferrara', age: 44 },
{ id: 8, lastName: 'Frances', firstName: 'Rossini', age: 36 },
{ id: 9, lastName: 'Roxie', firstName: 'Harvey', age: 65 },
]}
/>
```
<Table cols={[
{ key: 'id', name: 'ID' },
{ key: 'firstName', name: 'First name' },
{ key: 'lastName', name: 'Last name' },
{ key: 'age', name: 'Age' }
]} data={[
{ id: 1, lastName: 'Snow', firstName: 'Jon', age: 35 },
{ id: 2, lastName: 'Lannister', firstName: 'Cersei', age: 42 },
{ id: 3, lastName: 'Lannister', firstName: 'Jaime', age: 45 },
{ id: 4, lastName: 'Stark', firstName: 'Arya', age: 16 },
{ id: 7, lastName: 'Clifford', firstName: 'Ferrara', age: 44 },
{ id: 8, lastName: 'Frances', firstName: 'Rossini', age: 36 },
{ id: 9, lastName: 'Roxie', firstName: 'Harvey', age: 65 },
]}
/>
### Table from Raw CSV
You can also pass raw CSV as the content ...
```
<Table csv={`
Year,Temp Anomaly
1850,-0.418
2020,0.923
`} />
```
<Table csv={`
Year,Temp Anomaly,
1850,-0.418
2020,0.923
`} />
### Table from a URL
<Table url='https://raw.githubusercontent.com/datopian/data-literate/main/public/_files/HadCRUT.5.0.1.0.analysis.summary_series.global.annual.csv' />
```
<Table url='https://raw.githubusercontent.com/datopian/data-literate/main/public/_files/HadCRUT.5.0.1.0.analysis.summary_series.global.annual.csv' />
```
## Charts
You can create charts using a simple syntax.
### Line Chart
<LineChart data={
[
["1850",-0.41765878],
["1851",-0.2333498],
["1852",-0.22939907],
["1853",-0.27035445],
["1854",-0.29163003]
]
}
/>
```
<LineChart data={
[
["1850",-0.41765878],
["1851",-0.2333498],
["1852",-0.22939907],
["1853",-0.27035445],
["1854",-0.29163003]
]
}
/>
```
NB: we have quoted years as otherwise not interpreted as dates but as integers ...
### Vega and Vega Lite
You can using vega or vega-lite. Here's an example using vega-lite:
<VegaLite data={ { "table": [
{
"y": -0.418,
"x": 1850
},
{
"y": 0.923,
"x": 2020
}
]
}
} spec={
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"mark": "bar",
"data": {
"name": "table"
},
"encoding": {
"x": {
"field": "x",
"type": "ordinal"
},
"y": {
"field": "y",
"type": "quantitative"
}
}
}
} />
```jsx
<VegaLite data={ { "table": [
{
"y": -0.418,
"x": 1850
},
{
"y": 0.923,
"x": 2020
}
]
}
} spec={
{
"$schema": "https://vega.github.io/schema/vega-lite/v4.json",
"mark": "bar",
"data": {
"name": "table"
},
"encoding": {
"x": {
"field": "x",
"type": "ordinal"
},
"y": {
"field": "y",
"type": "quantitative"
}
}
}
} />
```
#### Line Chart from URL with Tooltip
https://vega.github.io/vega-lite/examples/interactive_multi_line_pivot_tooltip.html
<VegaLite spec={
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"data": {"url": "/_files/HadCRUT.5.0.1.0.analysis.summary_series.global.annual.csv"},
"width": 600,
"height": 250,
"mark": "line",
"encoding": {
"x": {"field": "Time", "type": "temporal"},
"y": {"field": "Anomaly (deg C)", "type": "quantitative"},
"tooltip": {"field": "Anomaly (deg C)", "type": "quantitative"}
}
}
} />
## Display Excel Files
Local file ...
```
<Excel src='/_files/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018-data.xlsx' />
```
<Excel src='/_files/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018-data.xlsx' />
Remote files work too (even without CORS) thanks to proxying:
```
<Excel src='https://github.com/datasets/awesome-data/files/6604635/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018-data.xlsx' />
```
<Excel src='https://github.com/datasets/awesome-data/files/6604635/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018-data.xlsx' />

13
site/content/docs/sidebar.json
View File

@ -23,5 +23,18 @@
"href": "/docs/deploying-your-portaljs-app"
}
]
},
{
"name": "Examples",
"children": [
{
"name": "Data Catalog w/ CKAN datasets",
"href": "/docs/examples/example-ckan"
},
{
"name": "Data Catalog w/ GitHub datasets",
"href": "/docs/examples/example-data-catalog"
}
]
}
]

5
site/content/gallery.md
View File

@ -1,5 +0,0 @@
---
title: Gallery
---
Come back soon!

26
site/pages/blog.tsx
View File

@ -1,6 +1,8 @@
import Layout from '@/components/Layout';
import computeFields from '@/lib/computeFields';
import clientPromise from '@/lib/mddb';
import { BlogsList, SimpleLayout } from '@flowershow/core';
import * as fs from 'fs';
export default function Blog({ blogs }) {
return (
@ -32,19 +34,29 @@ export async function getStaticProps() {
blogs = [...blogs, ...docs];
const blogsSorted = blogs.sort(
const blogsWithComputedFields = blogs.map(async (blog) => {
const source = fs.readFileSync(blog.file_path, { encoding: 'utf-8' });
return await computeFields({
frontMatter: blog.metadata,
urlPath: blog.url_path,
filePath: blog.file_path,
source,
});
});
const blogList = await Promise.all(blogsWithComputedFields);
const blogsSorted = blogList.sort(
(a, b) =>
new Date(b.metadata.date).getTime() - new Date(a.metadata.date).getTime()
new Date(b.metadata?.date).getTime() -
new Date(a.metadata?.date).getTime()
);
// Temporary, flowershow/BlogsList expects the contentlayer fields
const blogsObjects = blogsSorted.map((b) => {
return { ...b, ...b.metadata };
});
return {
props: {
blogs: blogsObjects,
blogs: blogsSorted,
},
};
}