knight/datahub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[website,#778][xs]: remove learn content folder

Browse Source
This commit is contained in:
deme 2023-04-21 14:50:03 -03:00
parent 23a0420fcb
commit 22b916ea37
3 changed files with 0 additions and 501 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

218
site/content/learn/ckan.md
View File

@ -1,218 +0,0 @@
Live DEMOs:
- https://catalog-portal-js.vercel.app
- https://ckan-enterprise-frontend.vercel.app/
## Create a Portal app for CKAN
To create a Portal app, run the following command in your terminal:
```console
npx create-next-app -e https://github.com/datopian/portal.js/tree/main/examples/ckan
```
> NB: Under the hood, this uses the tool called create-next-app, which bootstraps an app for you based on our CKAN example.
## Guide
### Styling 🎨
We use Tailwind as a CSS framework. Take a look at `/styles/globals.css` to see what we're importing from Tailwind bundle. You can also configure Tailwind using `tailwind.config.js` file.
Have a look at Next.js support of CSS and ways of writing CSS:
https://nextjs.org/docs/basic-features/built-in-css-support
### Backend
So far the app is running with mocked data behind. You can connect CMS and DMS backends easily via environment variables:
```console
$ export DMS=http://ckan:5000
$ export CMS=http://myblog.wordpress.com
```
> Note that we don't yet have implementations for the following CKAN features:
>
> - Activities
> - Auth
> - Groups
> - Facets
### Routes
These are the default routes set up in the "starter" app.
- Home `/`
- Search `/search`
- Dataset `/@org/dataset`
- Resource `/@org/dataset/r/resource`
- Organization `/@org`
- Collection (aka group in CKAN) (?) - suggest to merge into org
- Static pages, eg, `/about` etc. from CMS or can do it without external CMS, e.g., in Next.js.
### New Routes
You can create new routes in `/pages` directory where each file is associated with a route based on its name. We suggest using [Next.JS docs][] for more detailed information.
[next.js docs]: https://nextjs.org/docs/basic-features/pages
### Data fetching
We use Apollo client which allows us to query data with GraphQL. We have setup CKAN API for the demo (it uses demo.ckan.org as DMS):
http://portal.datopian1.now.sh/
Note that we don't have Apollo Server but we connect CKAN API using [`apollo-link-rest`](https://www.apollographql.com/docs/link/links/rest/) module. You can see how it works in [lib/apolloClient.ts](https://github.com/datopian/portal/blob/master/lib/apolloClient.ts) and then have a look at [pages/\_app.tsx](https://github.com/datopian/portal/blob/master/pages/_app.tsx).
For development/debugging purposes, we suggest installing the Chrome extension - https://chrome.google.com/webstore/detail/apollo-client-developer-t/jdkknkkbebbapilgoeccciglkfbmbnfm.
### I18n configuration
Portal.js is configured by default to support both `English` and `French` subpath for language translation. But for subsequent users, this following steps can be used to configure i18n for other languages;
1. Update `next.config.js`, to add more languages to the i18n locales
```js
i18n: {
locales: ['en', 'fr', 'nl-NL'], // add more language to the list
defaultLocale: 'en', // set the default language to use
},
```
2. Create a folder for the language in `locales` --> `locales/en-Us`
3. In the language folder, different namespace files (json) can be created for each translation. For the `index.js` use-case, I named it `common.json`
```json
// locales/en/common.json
{
"title" : "Portal js in English",
}
// locales/fr/common.json
{
"title" : "Portal js in French",
}
```
4. To use on pages using Server-side Props.
```js
import { loadNamespaces } from './_app';
import useTranslation from 'next-translate/useTranslation';
const Home: React.FC = ()=> {
const { t } = useTranslation();
return (
<div>{t(`common:title`)}</div> // we use common and title base on the common.json data
);
};
export const getServerSideProps: GetServerSideProps = async ({ locale }) => {
........ ........
return {
props : {
_ns: await loadNamespaces(['common'], locale),
}
};
};
```
5. Go to the browser and view the changes using language subpath like this `http://localhost:3000` and `http://localhost:3000/fr`. **Note** The subpath also activate chrome language Translator
### Pre-fetch data in the server-side
When visiting a dataset page, you may want to fetch the dataset metadata in the server-side. To do so, you can use `getServerSideProps` function from NextJS:
```javascript
import { GetServerSideProps } from 'next';
import { initializeApollo } from '../lib/apolloClient';
import gql from 'graphql-tag';
const QUERY = gql`
query dataset($id: String) {
dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") {
result
}
}
`;
...
export const getServerSideProps: GetServerSideProps = async (context) => {
const apolloClient = initializeApollo();
await apolloClient.query({
query: QUERY,
variables: {
id: 'my-dataset'
},
});
return {
props: {
initialApolloState: apolloClient.cache.extract(),
},
};
};
```
This would fetch the data from DMS and save it in the Apollo cache so that we can query it again from the components.
### Access data from a component
Consider situation when rendering a component for org info on the dataset page. We already have pre-fetched dataset metadata that includes `organization` property with attributes such as `name`, `title` etc. We can now query only organization part for our `Org` component:
```javascript
import { useQuery } from '@apollo/react-hooks';
import gql from 'graphql-tag';
export const GET_ORG_QUERY = gql`
query dataset($id: String) {
dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") {
result {
organization {
name
title
image_url
}
}
}
}
`;
export default function Org({ variables }) {
const { loading, error, data } = useQuery(
GET_ORG_QUERY,
{
variables: { id: 'my-dataset' }
}
);
...
const { organization } = data.dataset.result;
return (
<>
{organization ? (
<>
<img
src={
organization.image_url
}
className="h-5 w-5 mr-2 inline-block"
/>
<Link href={`/@${organization.name}`} className="font-semibold text-primary underline">
{organization.title || organization.name}
</Link>
</>
) : (
''
)}
</>
);
}
```

251
site/content/learn/deploy-to-gh-pages.md
View File

@ -1,251 +0,0 @@
# Deploying data on Github using Portal.js and Github pages
---
**Use Case:**
---
You have some data in a Github repo and you'd like to deploy it online using "portal" so that it is easy for others to view, explore and use.
---
Here we show how you can use portal.js plus github actions to deploy your dataset in minutes and keep it updated as you make changes.
The example focuses on the case of a [Frictionless dataset][fd] but it works for any dataset type supported by portal.js.
We provide three options on how to do this and recommend using the first one unless you really want to get hands on:
* Deploying datasets automatically by setting up a github actions script.
* Deploying datasets from a local bash script with portal code commits
* Deploying datasets from a local bash script without portal code commits
[fd]: https://frictionlessdata.io/data-packages/
## Deploy datasets automatically by setting up a github actions script
The github actions below will automatically build and deploy a single page, Frictionless dataset to `gh-pages` branch. Follow the steps below to achieve this:
1. Create a secret so we can automatically commit to gh-pages branch (see below)
2. Set up the github action to build portal to your dataset and deploy it (see below)
3. Wait for your page to build and then setup github pages (see below)
4. View the results: visit `https://<your github username>/github.io/<dataset repo name>/`
### Step 1
In the dataset repository you want to deploy, create a github secret with the name `PORTAL_REPO_NAME` and the value should be the name of the repository.
See steps on creating a secret [here](https://docs.github.com/en/actions/reference/encrypted-secrets)
<img src="/scripts/assets/secrets.png" />
### Step 2
In the dataset repository you want deploy create a `.github/workflow` directory and add a `main.yml` file with the following content (you can also view/download this [action file here](scripts/actions/single-dataset-ssg.yml):
```bash
name: github pages
on:
push:
branches:
- master
- main
jobs:
run:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Setup Node
uses: actions/setup-node@v2.1.2
with:
node-version: '12.x'
- name: Build datasets
env:
PORTAL_REPO_NAME: ${{ secrets.PORTAL_REPO_NAME }}
run: |
curl https://raw.githubusercontent.com/datopian/portal.js/main/site/public/scripts/single-dataset-no-commit.sh > portal.sh
git config --local user.email "$(git log --format='%ae' HEAD^!)"
git config --local user.name "$(git log --format='%an' HEAD^!)"
source ./portal.sh
```
Then, commit and push your code.
```bash
git add .
git commit -m "Build dataset page"
git push
```
### Step 3
Wait for a while as your page builds, and once you see the green check mark, navigate to your repository's github `pages` in settings, set the `source` to `gh-pages` and folder to `/root`:
<img src='/scripts/assets/sdnocommit.png' />
## Deploy single dataset without commiting portal.js code
Users who want to deploy datasets from a local bash script without saving/commiting the portal.js code, can use the script shown below.
Using this script means you do not have access to the portal.js code used to generate the dataset page, and as such cannot modify/extend it.
This script creates and commit only the build/output files to the gh-pages branch. Follow the steps below to achieve this.
### Step 1
Clone/Pull the dataset repository you want deploy. For example:
```bash
git clone https://github.com/datasets/finance-vix
cd finance-vix
```
### Step 2
In a terminal, export an env variable with the name of your dataset github repo. For example if deploying https://github.com/datasets/finance-vix, then export the name as:
```bash
export PORTAL_REPO_NAME=finance-vix
```
### Step 3
In the dataset repository's root folder, create a file called `portal.sh` and paste the following [content](/scripts/single-dataset-no-commit.sh):
```bash
#!/bin/bash
git checkout -b gh-pages
git rm -r --cached .
rm -rf portal
mkdir -p portal
npx create-next-app portal -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless
mkdir portal/public/dataset
cp -a ./data portal/public/dataset
cp -a ./datapackage.json portal/public/dataset
cp -a ./README.md portal/public/dataset
PORTAL_DATASET_PATH=$PWD"/portal/public/dataset"
export PORTAL_DATASET_PATH
cd portal
assetPrefix='"/'$PORTAL_REPO_NAME'/"'
basePath='"/'$PORTAL_REPO_NAME'"'
echo 'module.exports = {assetPrefix:' ${assetPrefix}', basePath: '${basePath}' }' > next.config.js ## This ensures css and public folder works
yarn export
cd ..
cp -R -a portal/out/* ./
touch .nojekyll
git add $PWD'/_next' $PWD'/index.html' $PWD'/dataset' $PWD'/404.html' $PWD'/.nojekyll' $PWD'/favicon.ico'
git commit -m "Build new dataset page"
git push origin gh-pages
```
### Step 4
Run the bash script in a terminal with:
```bash
source portal.sh
```
> Note: Use `source` instead of `bash` so that the script can work well with environment variables.
### Step 5
Go to your repository's github `pages` in setting and set the Branch to gh-pages and folder to root:
<img src='/scripts/assets/sdnocommit.png' />
### Step 6
Open your deployed site at `https://<your github username>/github.io/<dataset repo name>`
## Deploy single dataset with portal commit
Users who want access to the portal.js code used for generating the dataset page can use the script shown in the following section.
This script creates and commits the portal.js code to the root branch and also adds an automated script to deploy to gh-page. Follow the steps below to use this script.
### Step 1
Create a Github Personal Access Token (PAT). See steps [here](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token)
### Step 2
In the dataset repository you want to deploy, create a github secret with the name `PORTAL_NEXT_TOKEN`. The value should be the PAT created in step 1. See steps on creating a secret [here](https://docs.github.com/en/actions/reference/encrypted-secrets)
> Note: Without the PAT and the secret configured, the automatic build will fail.
### Step 3
Clone/Pull the dataset repository you want deploy. For example:
```bash
git clone https://github.com/datasets/finance-vix
cd finance-vix
```
### Step 4
In your computer's terminal/command prompt, export an environment variable with the name of your dataset's github repo.
For example if you want to deploy the dataset at https://github.com/datasets/finance-vix, then export the name using the command:
```bash
export PORTAL_REPO_NAME=finance-vix
```
### Step 5
Create a file called `portal.sh` and paste the following [content](/scripts/single-dataset-commit.sh):
```bash
#!/bin/bash
rm -rf portal
mkdir -p portal
npx create-next-app portal -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless
mkdir portal/public/dataset
cp -a ./data portal/public/dataset
cp -a ./datapackage.json portal/public/dataset
cp -a ./README.md portal/public/dataset
PORTAL_DATASET_PATH=$PWD"/portal/public/dataset"
export PORTAL_DATASET_PATH
mkdir -p .github && mkdir -p .github/workflows && touch .github/workflows/main.yml
curl https://raw.githubusercontent.com/datopian/portal.js/main/site/public/scripts/gh-page-builder-action.yml > .github/workflows/main.yml
cd portal
assetPrefix='"/'$PORTAL_REPO_NAME'/"'
basePath='"/'$PORTAL_REPO_NAME'"'
echo 'module.exports = {assetPrefix:' ${assetPrefix}', basePath: '${basePath}' }' > next.config.js ## This ensures css and public folder works
cd ..
git add .
git commit -m "Add dataset build feature"
git push
echo "Portal generated, please push your code to github"
```
### Step 6
Run the bash script with:
```bash
source portal.sh
```
> Note: Use `source` instead of `bash` so that the script can work well with environment variables.
### Step 7
Go to your repository's github `pages` in setting and set the Branch to gh-pages and folder to root:
<img src='/scripts/assets/sdnocommit.png' />
### Step 8
Open your deployed site at `https://<your github username>/github.io/<dataset repo name>`

32
site/content/learn/single-frictionless-dataset.md
View File

@ -1,32 +0,0 @@
Live Demo:
- https://portal-js.vercel.app/
## Create a single frictionless dataset portal
The dataset should be a frictionless dataset i.e. it should have a [datapackage.json](https://specs.frictionlessdata.io/data-package/)
Create a frictionless dataset portal app from the default template by executing the following command in your terminal:
```
$ npx create-next-app -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless
```
> Choose a name for your portal when prompted e.g. your-portal
Next, connect the frictionless dataset to `your-portal` by declaring the path to the directory level that contains the `datapackage.json` via an environment variable by executing the following command in your terminal:
```
$ cd your-portal
$ export PORTAL_DATASET_PATH=path/to/your/dataset
```
In `your-portal` directory, run the command below in your terminal to start the portal:
```
$ yarn dev
```
Open the page in your browser via the localhost url(usually http://localhost:3000) returned in the terminal to see your frictionless dataset portal.
### Styling 🎨
We use Tailwind as a CSS framework. Take a look at `/styles/tailwind.css` to see what we're importing from Tailwind bundle. You can also configure Tailwind using `tailwind.config.js` file.
Have a look at Next.js support of CSS and ways of writing CSS:
https://nextjs.org/docs/basic-features/built-in-css-support