Portal assumes a "decoupled" approach where the frontend is a separate service from the backend and interacts with backend(s) via an API. It can be used with any backend and has out of the box support for CKAN. portal is built in Javascript and React on top of the popular Next.js framework.

Live DEMO: https://portal.datopian1.now.sh

Features

🗺️ Unified sites: present data and content in one seamless site, pulling datasets from a DMS (e.g. CKAN) and content from a CMS (e.g. wordpress) with a common internal API.
👩‍💻 Developer friendly: built with familiar frontend tech Javascript, React etc
🔋 Batteries included: Full set of portal components out of the box e.g. catalog search, dataset showcase, blog etc.
🎨 Easy to theme and customize: installable themes, use standard CSS and React+CSS tooling. Add new routes quickly.
🧱E Extensible: quickly extend and develop/import your own React components
📝 Well documented: full set of documentation plus the documentation of NextJS and Apollo.

For developers

🏗 Build with modern, familiar frontend tech such as Javascript and React.
🚀 NextJS framework: so everything in NextJS for free React, SSR, static site generation, huge number of examples and integrations etc.
- SSR => unlimited number of pages, SEO etc whilst still using React.
- Static Site Generation (SSG) (good for small sites) => ultra-simple deployment, great performance and lighthouse scores etc
📋 Typescript support

Getting Started

Setup

Install a recent version of Node. You'll need Node 10.13 or later.

Create a Portal app

To create a Portal app, open your terminal, cd into the directory you'd like to create the app in, and run the following command:

npm init portal-app my-data-portal

NB: Under the hood, this uses the tool called create-next-app, which bootstraps a Next.js app for you. It uses this template through the --example flag.

If it doesn’t work, please open an issue.

Guide

Styling 🎨

We use Tailwind as a CSS framework. Take a look at /styles/index.css to see what we're importing from Tailwind bundle. You can also configure Tailwind using tailwind.config.js file.

Have a look at Next.js support of CSS and ways of writing CSS:

https://nextjs.org/docs/basic-features/built-in-css-support

Backend

So far the app is running with mocked data behind. You can connect CMS and DMS backends easily via environment variables:

$ export DMS=http://ckan:5000
$ export CMS=http://myblog.wordpress.com

Note that we don't yet have implementations for the following CKAN features:

Activities

Auth

Groups

Facets

Routes

These are the default routes set up in the "starter" app.

Home /
Search /search
Dataset /@org/dataset
Resource /@org/dataset/r/resource
Organization /@org
Collection (aka group in CKAN) (?) - suggest to merge into org
Static pages, eg, /about etc. from CMS or can do it without external CMS, e.g., in Next.js

New Routes

TODO

Data fetching

We use Apollo client which allows us to query data with GraphQL. We have setup CKAN API for the demo (it uses demo.ckan.org as DMS):

http://portal.datopian1.now.sh/

Note that we don't have Apollo Server but we connect CKAN API using apollo-link-rest module. You can see how it works in lib/apolloClient.ts and then have a look at pages/_app.tsx.

For development/debugging purposes, we suggest installing the Chrome extension - https://chrome.google.com/webstore/detail/apollo-client-developer-t/jdkknkkbebbapilgoeccciglkfbmbnfm.

Pre-fetch data in the server-side

When visiting a dataset page, you may want to fetch the dataset metadata in the server-side. To do so, you can use getServerSideProps function from NextJS:

import { GetServerSideProps } from 'next';
import { initializeApollo } from '../lib/apolloClient';
import gql from 'graphql-tag';

const QUERY = gql`
  query dataset($id: String) {
    dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") {
      result
    }
  }
`;

...

export const getServerSideProps: GetServerSideProps = async (context) => {
  const apolloClient = initializeApollo();

  await apolloClient.query({
    query: QUERY,
    variables: {
      id: 'my-dataset'
    },
  });

  return {
    props: {
      initialApolloState: apolloClient.cache.extract(),
    },
  };
};

This would fetch the data from DMS and save it in the Apollo cache so that we can query it again from the components.

Access data from a component

Consider situation when rendering a component for org info on the dataset page. We already have pre-fetched dataset metadata that includes organization property with attributes such as name, title etc. We can now query only organization part for our Org component:

import { useQuery } from '@apollo/react-hooks';
import gql from 'graphql-tag';

export const GET_ORG_QUERY = gql`
  query dataset($id: String) {
    dataset(id: $id) @rest(type: "Response", path: "package_show?{args}") {
      result {
        organization {
          name
          title
          image_url
        }
      }
    }
  }
`;

export default function Org({ variables }) {
  const { loading, error, data } = useQuery(
    GET_ORG_QUERY,
    {
      variables: { id: 'my-dataset' }
    }
  );

  ...

  const { organization } = data.dataset.result;

  return (
    <>
      {organization ? (
        <>
          <img
            src={
              organization.image_url
            }
            className="h-5 w-5 mr-2 inline-block"
          />
          <Link href={`/@${organization.name}`}>
            <a className="font-semibold text-primary underline">
              {organization.title || organization.name}
            </a>
          </Link>
        </>
      ) : (
        ''
      )}
    </>
  );
}

Add a new data source

TODO

Developers

Boot the local instance

Install the dependencies:

yarn # or npm i

Boot the demo portal:

$ yarn dev # or npm run dev

Open http://localhost:3000 to see the home page 🎉

You can start editing the page by modifying /pages/index.tsx. The page auto-updates as you edit the file.

Tests

We use Jest for running tests:

yarn test # or npm run test

# turn on watching
yarn test --watch

We use Cypress tests as well

yarn run e2e

Architecture

Language: Javascript
Framework: NextJS - https://nextjs.org/
Data layer API: GraphQL using Apollo. So controllers access data using GraphQL “gatsby like”

Key Pages

See https://tech.datopian.com/frontend/

README.md Unescape Escape

PORTAL.js The javascript framework for data portals