diff --git a/site/content/learn/ckan.md b/site/content/learn/ckan.md deleted file mode 100644 index 39df75ac..00000000 --- a/site/content/learn/ckan.md +++ /dev/null @@ -1,218 +0,0 @@ -Live DEMOs: - -- https://catalog-portal-js.vercel.app -- https://ckan-enterprise-frontend.vercel.app/ - -## Create a Portal app for CKAN - -To create a Portal app, run the following command in your terminal: - -```console -npx create-next-app -e https://github.com/datopian/portal.js/tree/main/examples/ckan -``` - -> NB: Under the hood, this uses the tool called create-next-app, which bootstraps an app for you based on our CKAN example. - -## Guide - -### Styling 🎨 - -We use Tailwind as a CSS framework. Take a look at `/styles/globals.css` to see what we're importing from Tailwind bundle. You can also configure Tailwind using `tailwind.config.js` file. - -Have a look at Next.js support of CSS and ways of writing CSS: - -https://nextjs.org/docs/basic-features/built-in-css-support - -### Backend - -So far the app is running with mocked data behind. You can connect CMS and DMS backends easily via environment variables: - -```console -$ export DMS=http://ckan:5000 -$ export CMS=http://myblog.wordpress.com -``` - -> Note that we don't yet have implementations for the following CKAN features: -> -> - Activities -> - Auth -> - Groups -> - Facets - -### Routes - -These are the default routes set up in the "starter" app. - -- Home `/` -- Search `/search` -- Dataset `/@org/dataset` -- Resource `/@org/dataset/r/resource` -- Organization `/@org` -- Collection (aka group in CKAN) (?) - suggest to merge into org -- Static pages, eg, `/about` etc. from CMS or can do it without external CMS, e.g., in Next.js. - -### New Routes - -You can create new routes in `/pages` directory where each file is associated with a route based on its name. We suggest using [Next.JS docs][] for more detailed information. - -[next.js docs]: https://nextjs.org/docs/basic-features/pages - -### Data fetching - -We use Apollo client which allows us to query data with GraphQL. We have setup CKAN API for the demo (it uses demo.ckan.org as DMS): - -http://portal.datopian1.now.sh/ - -Note that we don't have Apollo Server but we connect CKAN API using [`apollo-link-rest`](https://www.apollographql.com/docs/link/links/rest/) module. You can see how it works in [lib/apolloClient.ts](https://github.com/datopian/portal/blob/master/lib/apolloClient.ts) and then have a look at [pages/\_app.tsx](https://github.com/datopian/portal/blob/master/pages/_app.tsx). - -For development/debugging purposes, we suggest installing the Chrome extension - https://chrome.google.com/webstore/detail/apollo-client-developer-t/jdkknkkbebbapilgoeccciglkfbmbnfm. - -### I18n configuration - -Portal.js is configured by default to support both `English` and `French` subpath for language translation. But for subsequent users, this following steps can be used to configure i18n for other languages; - -1. Update `next.config.js`, to add more languages to the i18n locales - -```js -i18n: { - locales: ['en', 'fr', 'nl-NL'], // add more language to the list - defaultLocale: 'en', // set the default language to use -}, -``` - -2. Create a folder for the language in `locales` --> `locales/en-Us` - -3. In the language folder, different namespace files (json) can be created for each translation. For the `index.js` use-case, I named it `common.json` - -```json -// locales/en/common.json -{ - "title" : "Portal js in English", -} - -// locales/fr/common.json -{ - "title" : "Portal js in French", -} -``` - -4. To use on pages using Server-side Props. - -```js -import { loadNamespaces } from './_app'; -import useTranslation from 'next-translate/useTranslation'; - -const Home: React.FC = ()=> { - const { t } = useTranslation(); - return ( -
{t(`common:title`)}
// we use common and title base on the common.json data - ); -}; - -export const getServerSideProps: GetServerSideProps = async ({ locale }) => { - ........ ........ - return { - props : { - _ns: await loadNamespaces(['common'], locale), - } - }; -}; - -``` - -5. Go to the browser and view the changes using language subpath like this `http://localhost:3000` and `http://localhost:3000/fr`. **Note** The subpath also activate chrome language Translator - -### Pre-fetch data in the server-side - -When visiting a dataset page, you may want to fetch the dataset metadata in the server-side. To do so, you can use `getServerSideProps` function from NextJS: - -```javascript -import { GetServerSideProps } from 'next'; -import { initializeApollo } from '../lib/apolloClient'; -import gql from 'graphql-tag'; - -const QUERY = gql` - query dataset($id: String) { - dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") { - result - } - } -`; - -... - -export const getServerSideProps: GetServerSideProps = async (context) => { - const apolloClient = initializeApollo(); - - await apolloClient.query({ - query: QUERY, - variables: { - id: 'my-dataset' - }, - }); - - return { - props: { - initialApolloState: apolloClient.cache.extract(), - }, - }; -}; -``` - -This would fetch the data from DMS and save it in the Apollo cache so that we can query it again from the components. - -### Access data from a component - -Consider situation when rendering a component for org info on the dataset page. We already have pre-fetched dataset metadata that includes `organization` property with attributes such as `name`, `title` etc. We can now query only organization part for our `Org` component: - -```javascript -import { useQuery } from '@apollo/react-hooks'; -import gql from 'graphql-tag'; - -export const GET_ORG_QUERY = gql` - query dataset($id: String) { - dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") { - result { - organization { - name - title - image_url - } - } - } - } -`; - -export default function Org({ variables }) { - const { loading, error, data } = useQuery( - GET_ORG_QUERY, - { - variables: { id: 'my-dataset' } - } - ); - - ... - - const { organization } = data.dataset.result; - - return ( - <> - {organization ? ( - <> - - - {organization.title || organization.name} - - - ) : ( - '' - )} - - ); -} -``` diff --git a/site/content/learn/deploy-to-gh-pages.md b/site/content/learn/deploy-to-gh-pages.md deleted file mode 100644 index 36a6319c..00000000 --- a/site/content/learn/deploy-to-gh-pages.md +++ /dev/null @@ -1,251 +0,0 @@ -# Deploying data on Github using Portal.js and Github pages ---- -**Use Case:** ---- -You have some data in a Github repo and you'd like to deploy it online using "portal" so that it is easy for others to view, explore and use. ---- -Here we show how you can use portal.js plus github actions to deploy your dataset in minutes and keep it updated as you make changes. - -The example focuses on the case of a [Frictionless dataset][fd] but it works for any dataset type supported by portal.js. - -We provide three options on how to do this and recommend using the first one unless you really want to get hands on: - -* Deploying datasets automatically by setting up a github actions script. -* Deploying datasets from a local bash script with portal code commits -* Deploying datasets from a local bash script without portal code commits - -[fd]: https://frictionlessdata.io/data-packages/ - -## Deploy datasets automatically by setting up a github actions script - -The github actions below will automatically build and deploy a single page, Frictionless dataset to `gh-pages` branch. Follow the steps below to achieve this: - -1. Create a secret so we can automatically commit to gh-pages branch (see below) -2. Set up the github action to build portal to your dataset and deploy it (see below) -3. Wait for your page to build and then setup github pages (see below) -4. View the results: visit `https:///github.io//` - -### Step 1 - -In the dataset repository you want to deploy, create a github secret with the name `PORTAL_REPO_NAME` and the value should be the name of the repository. - -See steps on creating a secret [here](https://docs.github.com/en/actions/reference/encrypted-secrets) - - - -### Step 2 - -In the dataset repository you want deploy create a `.github/workflow` directory and add a `main.yml` file with the following content (you can also view/download this [action file here](scripts/actions/single-dataset-ssg.yml): - -```bash -name: github pages - -on: - push: - branches: - - master - - main - -jobs: - run: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v2 - - name: Setup Node - uses: actions/setup-node@v2.1.2 - with: - node-version: '12.x' - - - name: Build datasets - env: - PORTAL_REPO_NAME: ${{ secrets.PORTAL_REPO_NAME }} - run: | - curl https://raw.githubusercontent.com/datopian/portal.js/main/site/public/scripts/single-dataset-no-commit.sh > portal.sh - git config --local user.email "$(git log --format='%ae' HEAD^!)" - git config --local user.name "$(git log --format='%an' HEAD^!)" - source ./portal.sh - -``` - -Then, commit and push your code. - -```bash -git add . -git commit -m "Build dataset page" -git push -``` - -### Step 3 - -Wait for a while as your page builds, and once you see the green check mark, navigate to your repository's github `pages` in settings, set the `source` to `gh-pages` and folder to `/root`: - - - - -## Deploy single dataset without commiting portal.js code - -Users who want to deploy datasets from a local bash script without saving/commiting the portal.js code, can use the script shown below. - -Using this script means you do not have access to the portal.js code used to generate the dataset page, and as such cannot modify/extend it. - -This script creates and commit only the build/output files to the gh-pages branch. Follow the steps below to achieve this. - -### Step 1 - -Clone/Pull the dataset repository you want deploy. For example: - -```bash -git clone https://github.com/datasets/finance-vix -cd finance-vix -``` - -### Step 2 - -In a terminal, export an env variable with the name of your dataset github repo. For example if deploying https://github.com/datasets/finance-vix, then export the name as: - -```bash -export PORTAL_REPO_NAME=finance-vix -``` - -### Step 3 - -In the dataset repository's root folder, create a file called `portal.sh` and paste the following [content](/scripts/single-dataset-no-commit.sh): - -```bash -#!/bin/bash -git checkout -b gh-pages -git rm -r --cached . -rm -rf portal -mkdir -p portal -npx create-next-app portal -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless -mkdir portal/public/dataset - -cp -a ./data portal/public/dataset -cp -a ./datapackage.json portal/public/dataset -cp -a ./README.md portal/public/dataset - -PORTAL_DATASET_PATH=$PWD"/portal/public/dataset" -export PORTAL_DATASET_PATH - -cd portal -assetPrefix='"/'$PORTAL_REPO_NAME'/"' -basePath='"/'$PORTAL_REPO_NAME'"' -echo 'module.exports = {assetPrefix:' ${assetPrefix}', basePath: '${basePath}' }' > next.config.js ## This ensures css and public folder works -yarn export - -cd .. -cp -R -a portal/out/* ./ -touch .nojekyll -git add $PWD'/_next' $PWD'/index.html' $PWD'/dataset' $PWD'/404.html' $PWD'/.nojekyll' $PWD'/favicon.ico' -git commit -m "Build new dataset page" -git push origin gh-pages -``` - -### Step 4 - -Run the bash script in a terminal with: - -```bash -source portal.sh -``` - -> Note: Use `source` instead of `bash` so that the script can work well with environment variables. - -### Step 5 - -Go to your repository's github `pages` in setting and set the Branch to gh-pages and folder to root: - - - -### Step 6 - -Open your deployed site at `https:///github.io/` - - -## Deploy single dataset with portal commit - -Users who want access to the portal.js code used for generating the dataset page can use the script shown in the following section. - -This script creates and commits the portal.js code to the root branch and also adds an automated script to deploy to gh-page. Follow the steps below to use this script. - -### Step 1 - -Create a Github Personal Access Token (PAT). See steps [here](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) - -### Step 2 - -In the dataset repository you want to deploy, create a github secret with the name `PORTAL_NEXT_TOKEN`. The value should be the PAT created in step 1. See steps on creating a secret [here](https://docs.github.com/en/actions/reference/encrypted-secrets) - -> Note: Without the PAT and the secret configured, the automatic build will fail. - -### Step 3 - -Clone/Pull the dataset repository you want deploy. For example: - -```bash -git clone https://github.com/datasets/finance-vix -cd finance-vix -``` - -### Step 4 - -In your computer's terminal/command prompt, export an environment variable with the name of your dataset's github repo. - -For example if you want to deploy the dataset at https://github.com/datasets/finance-vix, then export the name using the command: - -```bash -export PORTAL_REPO_NAME=finance-vix -``` - -### Step 5 - -Create a file called `portal.sh` and paste the following [content](/scripts/single-dataset-commit.sh): - -```bash -#!/bin/bash -rm -rf portal -mkdir -p portal -npx create-next-app portal -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless -mkdir portal/public/dataset - -cp -a ./data portal/public/dataset -cp -a ./datapackage.json portal/public/dataset -cp -a ./README.md portal/public/dataset - -PORTAL_DATASET_PATH=$PWD"/portal/public/dataset" -export PORTAL_DATASET_PATH - -mkdir -p .github && mkdir -p .github/workflows && touch .github/workflows/main.yml -curl https://raw.githubusercontent.com/datopian/portal.js/main/site/public/scripts/gh-page-builder-action.yml > .github/workflows/main.yml - -cd portal -assetPrefix='"/'$PORTAL_REPO_NAME'/"' -basePath='"/'$PORTAL_REPO_NAME'"' -echo 'module.exports = {assetPrefix:' ${assetPrefix}', basePath: '${basePath}' }' > next.config.js ## This ensures css and public folder works - -cd .. -git add . -git commit -m "Add dataset build feature" -git push -echo "Portal generated, please push your code to github" -``` - -### Step 6 - -Run the bash script with: - -```bash -source portal.sh -``` - -> Note: Use `source` instead of `bash` so that the script can work well with environment variables. - -### Step 7 - -Go to your repository's github `pages` in setting and set the Branch to gh-pages and folder to root: - - - -### Step 8 - -Open your deployed site at `https:///github.io/` diff --git a/site/content/learn/single-frictionless-dataset.md b/site/content/learn/single-frictionless-dataset.md deleted file mode 100644 index dc623f95..00000000 --- a/site/content/learn/single-frictionless-dataset.md +++ /dev/null @@ -1,32 +0,0 @@ -Live Demo: -- https://portal-js.vercel.app/ - -## Create a single frictionless dataset portal - -The dataset should be a frictionless dataset i.e. it should have a [datapackage.json](https://specs.frictionlessdata.io/data-package/) - - -Create a frictionless dataset portal app from the default template by executing the following command in your terminal: -``` -$ npx create-next-app -e https://github.com/datopian/portal.js/tree/main/examples/dataset-frictionless -``` -> Choose a name for your portal when prompted e.g. your-portal - -Next, connect the frictionless dataset to `your-portal` by declaring the path to the directory level that contains the `datapackage.json` via an environment variable by executing the following command in your terminal: -``` -$ cd your-portal -$ export PORTAL_DATASET_PATH=path/to/your/dataset -``` -In `your-portal` directory, run the command below in your terminal to start the portal: -``` -$ yarn dev -``` -Open the page in your browser via the localhost url(usually http://localhost:3000) returned in the terminal to see your frictionless dataset portal. - -### Styling 🎨 - -We use Tailwind as a CSS framework. Take a look at `/styles/tailwind.css` to see what we're importing from Tailwind bundle. You can also configure Tailwind using `tailwind.config.js` file. - -Have a look at Next.js support of CSS and ways of writing CSS: - -https://nextjs.org/docs/basic-features/built-in-css-support