Openspending blog

packages/components/stories/FlatUiTable.stories.ts

@@ -64,6 +64,6 @@ export const FromRawCSV: Story = {
 export const FromURL: Story = {
   name: 'Table from URL',
   args: {
-    url: 'https://ckan-dev.sse.datopian.com/datastore/dump/601c9cf0-595e-46d8-88fc-d1ab2904e2db',
+    url: 'https://storage.openspending.org/alberta-budget/__os_imported__alberta_total.csv',
   },
 };

site/content/blog/markdowndb-launch.md

@@ -0,0 +1,148 @@
+---
+title: 'Announcing MarkdownDB: an open source tool to create an SQL API to your markdown files! 🚀'
+description: MarkdownDB - an open source library to transform markdown content into sql-queryable data. Build rich markdown-powered sites easily and reliably. New dedicated website at markdowndb.com
+date: 2023-10-11
+authors: ['Ola Rubaj']
+---
+
+Hello, dear readers!
+
+We're excited to announce the official launch of MarkdownDB, an open source library to transform markdown content into sql-queryable data. Get a rich SQL API to your markdown files in seconds and build rich markdown-powered sites easily and reliably.
+
+We've also created a new dedicated website:
+
+https://markdowndb.com/
+
+## 🔍 What is MarkdownDB?
+
+MarkdownDB transforms your Markdown content into a queryable, lightweight SQLite database. Imagine being able to treat your collection of markdown files like entries in a database! Ever thought about fetching documents with specific tags or created in the past week? Or, say, pulling up all tasks across documents marked with the "⏭️" emoji? With MarkdownDB, all of this (and more) is not just possible, but a breeze.
+
+## 🌟 Features
+
+- Rich metadata extracted including frontmatter, links and more.
+- Lightweight and fast indexing 1000s of files in seconds.
+- Open source and extensible via plugin system.
+
+## 🚀 How it works
+
+### You have a folder of markdown content
+
+For example, your blog posts. Each file can have a YAML frontmatter header with metadata like title, date, tags, etc.
+
+```md
+---
+title: My first blog post
+date: 2021-01-01
+tags: [a, b, c]
+author: John Doe
+---
+
+# My first blog post
+
+This is my first blog post.
+I'm using MarkdownDB to manage my blog posts.
+```
+
+### Index the files with MarkdownDB
+
+Use the npm `mddb` package to index Markdown files into an SQLite database. This will create a `markdown.db` file in the current directory.
+
+```bash
+# npx mddb <path-to-folder-with-your-md-files>
+npx mddb ./blog
+```
+
+### Query your files with SQL...
+
+E.g. get all the files with with tag `a`.
+
+```sql
+SELECT files.*
+FROM files
+INNER JOIN file_tags ON files._id = file_tags.file
+WHERE file_tags.tag = 'a'
+```
+
+### ...or using MarkdownDB Node.js API in a framework of your choice!
+
+Use our Node API to query your data for your blog, wiki, docs, digital garden, or anything you want!
+
+E.g. here is an example of a Next.js page:
+
+```js
+// @/pages/blog/index.js
+import React from 'react';
+import clientPromise from '@/lib/mddb.mjs';
+
+export default function Blog({ blogs }) {
+  return (
+    <div>
+      <h1>Blog</h1>
+      <ul>
+        {blogs.map((blog) => (
+          <li key={blog.id}>
+            <a href={blog.url_path}>{blog.title}</a>
+          </li>
+        ))}
+      </ul>
+    </div>
+  );
+}
+
+export const getStaticProps = async () => {
+  const mddb = await clientPromise;
+  // get all files that are not marked as draft in the frontmatter
+  const blogFiles = await mddb.getFiles({
+    frontmatter: {
+      draft: false,
+    },
+  });
+
+  const blogsList = blogFiles.map(({ metadata, url_path }) => ({
+    ...metadata,
+    url_path,
+  }));
+
+  return {
+    props: {
+      blogs: blogsList,
+    },
+  };
+};
+```
+
+And the imported library with MarkdownDB database connection:
+
+```js
+// @/lib/mddb.mjs
+import { MarkdownDB } from 'mddb';
+
+const dbPath = 'markdown.db';
+
+const client = new MarkdownDB({
+  client: 'sqlite3',
+  connection: {
+    filename: dbPath,
+  },
+});
+
+const clientPromise = client.init();
+
+export default clientPromise;
+```
+
+### Build your blog, wiki, docs, digital garden, or anything you want
+
+And share it with the world!
+
+![[markdowbdb-launch-site-example.png]]
+
+👉 [Read the full tutorial](https://markdowndb.com/blog/basic-tutorial)
+
+---
+
+Find out more on our new official website: https://markdowndb.com/
+
+Check out the source on GitHub: https://github.com/datopian/markdowndb
+
+— Ola Rubaj, Developer at Datopian

site/content/blog/openspending-revamp.md

@@ -0,0 +1,27 @@
+---
+title: The Openspending Revamp
+date: 2023-10-12
+authors: ['Luccas Mateus', 'João Demenech']
+filetype: 'blog'
+---
+
+Following up [our previous blog post about the Open Spending revamp](), we now want to share the underlying technologies and tools we used to achieve our goals. 
+
+First of all, we used PortalJS, a JavaScript library that provides a set of reusable React components for mainly for building data portals with data visualization. This is the core technology underlying the Open Spending website, and can be seen in action, for example, on the CSV previews powered by the [FlatUI Component](https://storybook.portaljs.org/?path=/story/components-flatuitable--from-url) from PortalJS. For more information, visit the official website at https://portaljs.org/. 
+
+![](https://hackmd.io/_uploads/Sypq8irW6.png)
+
+Following up on [our previous blog post about the Open Spending revamp](https://www.datopian.com/blog/the-open-spending-revamp), we now want to share the underlying technologies and tools we used to achieve our goals. 
+
+First, we used PortalJS, a JavaScript library that provides a set of reusable React components primarily for building data portals with data visualization. This is the core technology behind the Open Spending website and can be seen in action, for example, in the CSV previews powered by PortalJS' [FlatUI Component](https://storybook.portaljs.org/?path=/story/components-flatuitable--from-url). More information can be found on the official website at https://portaljs.org/. 
+
+![](https://hackmd.io/_uploads/rkUAvjBZp.png)
+![](https://hackmd.io/_uploads/H1CjUjSWp.png)
+
+Secondly, we stored the metadata as frictionless datapackages in the `os-data` GitHub organization. Frictionless datapackages are a simple format and standard for describing and packaging a collection of (in our case tabular) data. They are typically used to publish FAIR and open datasets. To learn more about frictionless datapackages, visit their documentation page at https://framework.frictionlessdata.io/.
+
+Octokit, a GitHub API client for Node.js, was used to easily retrieve the metadata for the datasets from the repositories. You can find more information about Octokit on its GitHub repository: https://github.com/octokit/octokit.js, and you can find all the dataset repositories on https://github.com/os-data.
+
+Finally, to store the data, we used Cloudflare R2, a cloud storage service that allows developers to store large amounts of blob data without the costly egress bandwidth fees associated with typical cloud storage services. We chose this product because it offers a generous free tier that we could use for our project. You can read more about Cloudflare R2 on their official announcement page at https://cloudflare.net/news/news-details/2021/Cloudflare-Announces-R2-Storage-Rapid-and-Reliable-S3-Compatible-Object-Storage-Designed-for-the-Edge/default.aspx.
+
+You can check out the code for this project at https://github.com/datopian/portaljs/tree/main/examples/openspending. The website is live on [openspending.org](https://www.openspending.org/), you can file issues at https://github.com/datopian/portaljs/issues and get help by accessing our [Discord Channel](https://discord.com/invite/EeyfGrGu4U).