Compare commits

..

3 Commits

Author SHA1 Message Date
Ola Rubaj
bfbf0f5304 [site/blog][s]: content adjustments 2023-10-12 11:08:38 +02:00
Ola Rubaj
0db2055da3 [site/blog][xs]: add emoji to title 2023-10-11 17:42:30 +02:00
Ola Rubaj
d96d5555a0 [site/blog][s]: markdowndb launch post 2023-10-11 17:41:24 +02:00
50 changed files with 397 additions and 5122 deletions

8
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,8 @@
{
"recommendations": [
"nrwl.angular-console",
"esbenp.prettier-vscode",
"firsttris.vscode-jest-runner",
"dbaeumer.vscode-eslint"
]
}

View File

@@ -1,56 +1,31 @@
<h1 align="center">
<a href="https://datahub.io/">
<img alt="datahub" src="http://datahub.io/datahub-cube.svg" width="146">
</a>
🌀 Portal.JS
<br />
Rapidly build rich data portals using a modern frontend framework
</h1>
<p align="center">
Bugs, issues and suggestions re DataHub Cloud ☁️ and DataHub OpenSource 🌀
<br />
<br /><a href="https://discord.gg/xfFDMPU9dC"><img src="https://dcbadge.vercel.app/api/server/xfFDMPU9dC" /></a>
</p>
* [What is Portal.JS ?](#What-is-Portal.JS)
* [Features](#Features)
* [For developers](#For-developers)
* [Docs](#Docs)
* [Community](#Community)
* [Appendix](#Appendix)
* [What happened to Recline?](#What-happened-to-Recline?)
## DataHub
# What is Portal.JS
This repo and issue tracker are for
🌀 Portal.JS is a framework for rapidly building rich data portal frontends using a modern frontend approach. Portal.JS can be used to present a single dataset or build a full-scale data catalog/portal.
- DataHub Cloud ☁️ - https://datahub.io/
- DataHub 🌀 - https://datahub.io/opensource
Built in JavaScript and React on top of the popular [Next.js](https://nextjs.com/) framework. Portal.JS assumes a "decoupled" approach where the frontend is a separate service from the backend and interacts with backend(s) via an API. It can be used with any backend and has out of the box support for [CKAN](https://ckan.org/).
### Issues
Found a bug: 👉 https://github.com/datopian/datahub/issues/new
### Discussions
Got a suggestion, a question, want some support or just want to shoot the breeze 🙂
Head to the discussion forum: 👉 https://github.com/datopian/datahub/discussions
### Chat on Discord
If you would prefer to get help via live chat check out our discord 👉
[Discord](https://discord.gg/xfFDMPU9dC)
### Docs
https://datahub.io/docs
## DataHub OpenSource 🌀
DataHub 🌀 is a platform for rapidly creating rich data portal and publishing systems using a modern frontend approach. Datahub can be used to publish a single dataset or build a full-scale data catalog/portal.
DataHub is built in JavaScript and React on top of the popular [Next.js](https://nextjs.com/) framework. DataHub assumes a "decoupled" approach where the frontend is a separate service from the backend and interacts with backend(s) via an API. It can be used with any backend and has out of the box support for [CKAN](https://ckan.org/), GitHub, Frictionless Data Packages and more.
### Features
## Features
- 🗺️ Unified sites: present data and content in one seamless site, pulling datasets from a DMS (e.g. CKAN) and content from a CMS (e.g. Wordpress) with a common internal API.
- 👩‍💻 Developer friendly: built with familiar frontend tech (JavaScript, React, Next.js).
- 🔋 Batteries included: full set of portal components out of the box e.g. catalog search, dataset showcase, blog, etc.
- 🎨 Easy to theme and customize: installable themes, use standard CSS and React+CSS tooling. Add new routes quickly.
- 🧱 Extensible: quickly extend and develop/import your own React components
- 📝 Well documented: full set of documentation plus the documentation of Next.js.
- 📝 Well documented: full set of documentation plus the documentation of Next.js and Apollo.
### For developers
@@ -58,3 +33,25 @@ DataHub is built in JavaScript and React on top of the popular [Next.js](https:/
- 🚀 Next.js framework: so everything in Next.js for free: Server Side Rendering, Static Site Generation, huge number of examples and integrations, etc.
- Server Side Rendering (SSR) => Unlimited number of pages, SEO and more whilst still using React.
- Static Site Generation (SSG) => Ultra-simple deployment, great performance, great lighthouse scores and more (good for small sites)
#### **Check out the [Portal.JS website](https://portaljs.org/) for a gallery of live portals**
___
# Docs
Access the Portal.JS documentation at:
https://portaljs.org/docs
- [Examples](https://portaljs.org/docs#examples)
# Community
If you have questions about anything related to Portal.JS, you're always welcome to ask our community on [GitHub Discussions](https://github.com/datopian/portal.js/discussions) or on our [Discord server](https://discord.gg/EeyfGrGu4U).
# Appendix
## What happened to Recline?
Portal.JS used to be Recline(JS). If you are looking for the old Recline codebase it still exists: see the [`recline` branch](https://github.com/datopian/portal.js/tree/recline). If you want context for the rename see [this issue](https://github.com/datopian/portal.js/issues/520).

View File

@@ -1,9 +1,3 @@
# PortalJS Demo replicating the FiveThirtyEight data portal
## 👉 https://fivethirtyeight.portaljs.org 👈
Here's a blog post we wrote about it: https://www.datopian.com/blog/fivethirtyeight-replica
This is a replica of the awesome data.fivethirtyeight.com using PortalJS.
You might be asking why we did that, there are three main reasons:

3522
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,79 +1,5 @@
# @portaljs/components
## 0.6.0
### Minor Changes
- [`a044f56e`](https://github.com/datopian/portaljs/commit/a044f56e3cbe0519ddf9d24d78b0bb7eac917e1c) Thanks [@luccasmmg](https://github.com/luccasmmg)! - Added plotly components
## 0.5.10
### Patch Changes
- [#1083](https://github.com/datopian/portaljs/pull/1083) [`86a2945e`](https://github.com/datopian/portaljs/commit/86a2945ee68dfcea0299984ca9cc9070d68fe1c2) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Created integration with datastore api for table component
## 0.5.9
### Patch Changes
- [#1081](https://github.com/datopian/portaljs/pull/1081) [`2bbf3134`](https://github.com/datopian/portaljs/commit/2bbf3134896df3ecc66560bdf95bece143614c7b) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Fixed error to remove anchor from document
## 0.5.8
### Patch Changes
- [#1079](https://github.com/datopian/portaljs/pull/1079) [`058d2367`](https://github.com/datopian/portaljs/commit/058d23678a024890f8a6d909ded9fc8fc11cf145) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Changed the download behaviour of the bucket viewer component and removed loading component while downloading
## 0.5.7
### Patch Changes
- [#1077](https://github.com/datopian/portaljs/pull/1077) [`6d7acd27`](https://github.com/datopian/portaljs/commit/6d7acd27ed9299cbcc14eab906f2f0eb414656b8) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Created property to present a component while is loading the download of the file and fixed download bug on pagination
## 0.5.6
### Patch Changes
- [#1075](https://github.com/datopian/portaljs/pull/1075) [`26dcffc2`](https://github.com/datopian/portaljs/commit/26dcffc279057f80a579134e862085ba042c06c3) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Fixed problem presenting the download component in the first load of the bucket viewer
## 0.5.5
### Patch Changes
- [#1073](https://github.com/datopian/portaljs/pull/1073) [`cf24042a`](https://github.com/datopian/portaljs/commit/cf24042a910567e98eeb75ade42ce0149bdb62d1) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Fixed filter by startDate error
## 0.5.4
### Patch Changes
- [#1071](https://github.com/datopian/portaljs/pull/1071) [`27c99add`](https://github.com/datopian/portaljs/commit/27c99adde8fa36ad2c2e03f227f93aa62454eefa) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Added pagination and filter properties for the BucketViewer component
## 0.5.3
### Patch Changes
- [#1066](https://github.com/datopian/portaljs/pull/1066) [`dd03a493`](https://github.com/datopian/portaljs/commit/dd03a493beca5459d1ef447b2df505609fc64e95) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Created Iframe component
## 0.5.2
### Patch Changes
- [#1063](https://github.com/datopian/portaljs/pull/1063) [`b13e3ade`](https://github.com/datopian/portaljs/commit/b13e3ade3ccefe7dffe84f824bdedd3e512ce499) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Created auto zoom configuration for the map component
## 0.5.1
### Patch Changes
- [#1061](https://github.com/datopian/portaljs/pull/1061) [`4ddfc112`](https://github.com/datopian/portaljs/commit/4ddfc1126a3f0b8137ea47a08a36c56b7373b8f6) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Created the style property in the Map component
## 0.5.0
### Minor Changes
- [#1055](https://github.com/datopian/portaljs/pull/1055) [`712f4a3b`](https://github.com/datopian/portaljs/commit/712f4a3b0f074e654879bb75059f51e06b422b32) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Creation of BucketViewer component to show the data of public buckets
- [#1057](https://github.com/datopian/portaljs/pull/1057) [`61c750b7`](https://github.com/datopian/portaljs/commit/61c750b7e11fe52bf04d25f192440ee1bb307404) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Exporting BucketViewer to be accessed out of the folder
## 0.4.0
### Minor Changes

View File

@@ -1,6 +1,6 @@
{
"name": "@portaljs/components",
"version": "0.6.0",
"version": "0.4.0",
"type": "module",
"description": "https://portaljs.org",
"keywords": [
@@ -29,8 +29,6 @@
"@githubocto/flat-ui": "^0.14.1",
"@heroicons/react": "^2.0.17",
"@planet/maps": "^8.1.0",
"@react-pdf-viewer/core": "3.6.0",
"@react-pdf-viewer/default-layout": "3.6.0",
"@tanstack/react-table": "^8.8.5",
"ag-grid-react": "^30.0.4",
"chroma-js": "^2.4.2",
@@ -39,19 +37,19 @@
"next-mdx-remote": "^4.4.1",
"ol": "^7.4.0",
"papaparse": "^5.4.1",
"pdfjs-dist": "2.15.349",
"plotly.js": "^2.30.1",
"postcss-url": "^10.1.3",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-hook-form": "^7.43.9",
"react-leaflet": "^4.2.1",
"react-plotly.js": "^2.6.0",
"react-query": "^3.39.3",
"react-vega": "^7.6.0",
"vega": "5.25.0",
"vega-lite": "5.1.0",
"vitest": "^0.31.4",
"@react-pdf-viewer/core": "3.6.0",
"@react-pdf-viewer/default-layout": "3.6.0",
"pdfjs-dist": "2.15.349",
"xlsx": "^0.18.5"
},
"devDependencies": {

View File

@@ -1,222 +0,0 @@
import { CSSProperties, ReactNode, useEffect, useState } from 'react';
import LoadingSpinner from './LoadingSpinner';
export interface BucketViewerFilterSearchedDataEvent {
startDate?: Date;
endDate?: Date;
}
export interface BucketViewerProps {
onLoadTotalNumberOfItems?: (total: number) => void;
domain: string;
downloadConfig?: {
hoverOfTheFileComponent?: ReactNode;
};
suffix?: string;
className?: string;
paginationConfig?: BucketViewerPaginationConfig;
filterState?: BucketViewerFilterSearchedDataEvent;
dataMapperFn: (rawData: Response) => Promise<BucketViewerData[]>;
}
export interface BucketViewerPaginationConfig {
containerClassName?: string;
containerStyles?: CSSProperties;
itemsPerPage: number;
}
export interface BucketViewerData {
fileName: string;
downloadFileUri: string;
dateProps?: {
date: Date;
dateFormatter?: (date: Date) => string;
};
}
export function BucketViewer({
domain,
suffix,
dataMapperFn,
className,
filterState,
paginationConfig,
downloadConfig,
onLoadTotalNumberOfItems,
}: BucketViewerProps) {
suffix = suffix ?? '/';
const { hoverOfTheFileComponent } = downloadConfig ?? {};
const [isLoading, setIsLoading] = useState<boolean>(false);
const [showDownloadComponentOnLine, setShowDownloadComponentOnLine] =
useState(-1);
const [currentPage, setCurrentPage] = useState<number>(0);
const [lastPage, setLastPage] = useState<number>(0);
const [bucketFiles, setBucketFiles] = useState<BucketViewerData[]>([]);
const [paginatedData, setPaginatedData] = useState<BucketViewerData[]>([]);
const [filteredData, setFilteredData] = useState<BucketViewerData[]>([]);
useEffect(() => {
setIsLoading(true);
fetch(`${domain}${suffix}`)
.then((res) => dataMapperFn(res))
.then((data) => {
setBucketFiles(data);
setFilteredData(data);
})
.finally(() => setIsLoading(false));
}, [domain, suffix]);
useEffect(() => {
if (paginationConfig) {
const startIndex = paginationConfig
? currentPage * paginationConfig.itemsPerPage
: 0;
const endIndex = paginationConfig
? startIndex + paginationConfig.itemsPerPage
: 0;
setLastPage(
Math.ceil(filteredData.length / paginationConfig.itemsPerPage) - 1
);
setPaginatedData(filteredData.slice(startIndex, endIndex));
}
}, [currentPage, filteredData]);
useEffect(() => {
if (onLoadTotalNumberOfItems) onLoadTotalNumberOfItems(filteredData.length);
}, [filteredData]);
useEffect(() => {
if (!filterState) return;
if (filterState.startDate && filterState.endDate) {
setFilteredData(
bucketFiles.filter(({ dateProps }) =>
dateProps
? dateProps.date.getTime() >= filterState.startDate.getTime() &&
dateProps.date.getTime() <= filterState.endDate.getTime()
: true
)
);
} else if (filterState.startDate) {
setFilteredData(
bucketFiles.filter(({ dateProps }) =>
dateProps
? dateProps.date.getTime() >= filterState.startDate.getTime()
: true
)
);
} else if (filterState.endDate) {
setFilteredData(
bucketFiles.filter(({ dateProps }) =>
dateProps
? dateProps.date.getTime() <= filterState.endDate.getTime()
: true
)
);
} else {
setFilteredData(bucketFiles);
}
}, [filterState]);
return isLoading ? (
<div className="w-full flex items-center justify-center h-[300px]">
<LoadingSpinner />
</div>
) : bucketFiles ? (
<>
{...(paginationConfig && bucketFiles ? paginatedData : filteredData)?.map(
(data, i) => (
<ul
onClick={() => {
const a: HTMLAnchorElement = document.createElement('a');
a.href = data.downloadFileUri;
a.target = `_blank`;
a.download = data.fileName;
document.body.appendChild(a);
a.click();
document.body.removeChild(a);
}}
key={i}
onMouseEnter={() => setShowDownloadComponentOnLine(i)}
onMouseLeave={() => setShowDownloadComponentOnLine(undefined)}
className={`${
className ??
'mb-2 border-b-[2px] border-b-[red] hover:cursor-pointer'
}`}
>
{hoverOfTheFileComponent && showDownloadComponentOnLine === i ? (
hoverOfTheFileComponent
) : (
<></>
)}
<div className="flex justify-between w-full items-center">
<div>
<li>{data.fileName}</li>
{data.dateProps && data.dateProps.dateFormatter ? (
<li>{data.dateProps.dateFormatter(data.dateProps.date)}</li>
) : (
<></>
)}
</div>
</div>
</ul>
)
)}
{paginationConfig ? (
<ul
className={
paginationConfig.containerClassName
? paginationConfig.containerClassName
: 'flex justify-end gap-x-[0.5rem] w-full'
}
style={paginationConfig.containerStyles ?? {}}
>
<li>
<button
className="hover:cursor-pointer hover:disabled:cursor-not-allowed"
disabled={currentPage === 0}
onClick={() => setCurrentPage(0)}
>
First
</button>
</li>
<li>
<button
className="hover:cursor-pointer hover:disabled:cursor-not-allowed"
onClick={() => setCurrentPage(currentPage - 1)}
disabled={currentPage === 0}
>
Previous
</button>
</li>
<label>{currentPage + 1}</label>
<li>
<button
onClick={() => setCurrentPage(currentPage + 1)}
disabled={currentPage >= lastPage}
className="hover:cursor-pointer hover:disabled:cursor-not-allowed"
>
Next
</button>
</li>
<li>
<button
onClick={() => setCurrentPage(lastPage)}
disabled={currentPage >= lastPage}
className="hover:cursor-pointer hover:disabled:cursor-not-allowed"
>
Last
</button>
</li>
</ul>
) : (
<></>
)}
</>
) : null;
}

View File

@@ -1,14 +0,0 @@
import { CSSProperties } from "react";
export interface IframeProps {
url: string;
style?: CSSProperties;
}
export function Iframe({
url, style
}: IframeProps) {
return (
<iframe src={url} style={style ?? { width: `100%`, height: `100%` }}></iframe>
);
}

View File

@@ -1,4 +1,4 @@
import { CSSProperties, useEffect, useState } from 'react';
import { useEffect, useState } from 'react';
import LoadingSpinner from './LoadingSpinner';
import loadData from '../lib/loadData';
import chroma from 'chroma-js';
@@ -30,10 +30,6 @@ export type MapProps = {
title?: string;
center?: { latitude: number | undefined; longitude: number | undefined };
zoom?: number;
style?: CSSProperties;
autoZoomConfiguration?: {
layerName: string
}
};
export function Map({
@@ -48,8 +44,6 @@ export function Map({
center = { latitude: 45, longitude: 45 },
zoom = 2,
title = '',
style = {},
autoZoomConfiguration = undefined,
}: MapProps) {
const [isLoading, setIsLoading] = useState<boolean>(false);
const [layersData, setLayersData] = useState<any>([]);
@@ -102,7 +96,6 @@ export function Map({
zoom={zoom}
scrollWheelZoom={false}
className="h-80 w-full"
style={style ?? {}}
// @ts-ignore
whenReady={(map: any) => {
// Enable zoom using scroll wheel
@@ -111,35 +104,17 @@ export function Map({
// Create the title box
var info = new L.Control() as any;
info.onAdd = function() {
info.onAdd = function () {
this._div = L.DomUtil.create('div', 'info');
this.update();
return this._div;
};
info.update = function() {
info.update = function () {
this._div.innerHTML = `<h4 style="font-weight: 600; background: #f9f9f9; padding: 5px; border-radius: 5px; color: #464646;">${title}</h4>`;
};
if (title) info.addTo(map.target);
if(!autoZoomConfiguration) return;
let layerToZoomBounds = L.latLngBounds(L.latLng(0, 0), L.latLng(0, 0));
layers.forEach((layer) => {
if(layer.name === autoZoomConfiguration.layerName) {
const data = layersData.find(
(layerData) => layerData.name === layer.name
)?.data;
if (data) {
layerToZoomBounds = L.geoJSON(data).getBounds();
return;
}
}
});
map.target.fitBounds(layerToZoomBounds);
}}
>
<TileLayer

View File

@@ -1,9 +0,0 @@
import Plot, { PlotParams } from "react-plotly.js";
export const Plotly: React.FC<PlotParams> = (props) => {
return (
<div>
<Plot {...props} />
</div>
);
};

View File

@@ -1,157 +0,0 @@
import { QueryClient, QueryClientProvider, useQuery } from "react-query";
import { Plotly } from "./Plotly";
import Papa, { ParseConfig } from "papaparse";
import LoadingSpinner from "./LoadingSpinner";
const queryClient = new QueryClient();
async function getCsv(url: string, bytes: number) {
const response = await fetch(url, {
headers: {
Range: `bytes=0-${bytes}`,
},
});
const data = await response.text();
return data;
}
async function parseCsv(
file: string,
parsingConfig: ParseConfig,
): Promise<any> {
return new Promise((resolve, reject) => {
Papa.parse(file, {
...parsingConfig,
header: true,
dynamicTyping: true,
skipEmptyLines: true,
transform: (value: string): string => {
return value.trim();
},
complete: (results: any) => {
return resolve(results);
},
error: (error: any) => {
return reject(error);
},
});
});
}
export interface PlotlyBarChartProps {
url?: string;
data?: { [key: string]: number | string }[];
rawCsv?: string;
randomId?: number;
bytes?: number;
parsingConfig?: ParseConfig;
xAxis: string;
yAxis: string;
lineLabel?: string;
title?: string;
}
export const PlotlyBarChart: React.FC<PlotlyBarChartProps> = ({
url,
data,
rawCsv,
bytes = 5132288,
parsingConfig = {},
xAxis,
yAxis,
lineLabel,
title = "",
}) => {
const randomId = Math.random();
return (
// Provide the client to your App
<QueryClientProvider client={queryClient}>
<PlotlyBarChartInner
url={url}
data={data}
rawCsv={rawCsv}
randomId={randomId}
bytes={bytes}
parsingConfig={parsingConfig}
xAxis={xAxis}
yAxis={yAxis}
lineLabel={lineLabel ?? yAxis}
title={title}
/>
</QueryClientProvider>
);
};
const PlotlyBarChartInner: React.FC<PlotlyBarChartProps> = ({
url,
data,
rawCsv,
randomId,
bytes,
parsingConfig,
xAxis,
yAxis,
lineLabel,
title,
}) => {
if (data) {
return (
<div className="w-full" style={{ height: "500px" }}>
<Plotly
layout={{
title,
}}
data={[
{
x: data.map((d) => d[xAxis]),
y: data.map((d) => d[yAxis]),
type: "bar",
name: lineLabel,
},
]}
/>
</div>
);
}
const { data: csvString, isLoading: isDownloadingCSV } = useQuery(
["dataCsv", url, randomId],
() => getCsv(url as string, bytes ?? 5132288),
{ enabled: !!url },
);
const { data: parsedData, isLoading: isParsing } = useQuery(
["dataPreview", csvString, randomId],
() =>
parseCsv(
rawCsv ? (rawCsv as string) : (csvString as string),
parsingConfig ?? {},
),
{ enabled: rawCsv ? true : !!csvString },
);
if (isParsing || isDownloadingCSV)
<div className="w-full flex justify-center items-center h-[500px]">
<LoadingSpinner />
</div>;
if (parsedData)
return (
<div className="w-full" style={{ height: "500px" }}>
<Plotly
layout={{
title,
}}
data={[
{
x: parsedData.data.map((d: any) => d[xAxis]),
y: parsedData.data.map((d: any) => d[yAxis]),
type: "bar",
name: lineLabel,
},
]}
/>
</div>
);
return (
<div className="w-full flex justify-center items-center h-[500px]">
<LoadingSpinner />
</div>
);
};

View File

@@ -1,157 +0,0 @@
import { QueryClient, QueryClientProvider, useQuery } from "react-query";
import { Plotly } from "./Plotly";
import Papa, { ParseConfig } from "papaparse";
import LoadingSpinner from "./LoadingSpinner";
const queryClient = new QueryClient();
async function getCsv(url: string, bytes: number) {
const response = await fetch(url, {
headers: {
Range: `bytes=0-${bytes}`,
},
});
const data = await response.text();
return data;
}
async function parseCsv(
file: string,
parsingConfig: ParseConfig,
): Promise<any> {
return new Promise((resolve, reject) => {
Papa.parse(file, {
...parsingConfig,
header: true,
dynamicTyping: true,
skipEmptyLines: true,
transform: (value: string): string => {
return value.trim();
},
complete: (results: any) => {
return resolve(results);
},
error: (error: any) => {
return reject(error);
},
});
});
}
export interface PlotlyLineChartProps {
url?: string;
data?: { [key: string]: number | string }[];
rawCsv?: string;
randomId?: number;
bytes?: number;
parsingConfig?: ParseConfig;
xAxis: string;
yAxis: string;
lineLabel?: string;
title?: string;
}
export const PlotlyLineChart: React.FC<PlotlyLineChartProps> = ({
url,
data,
rawCsv,
bytes = 5132288,
parsingConfig = {},
xAxis,
yAxis,
lineLabel,
title = "",
}) => {
const randomId = Math.random();
return (
// Provide the client to your App
<QueryClientProvider client={queryClient}>
<LineChartInner
url={url}
data={data}
rawCsv={rawCsv}
randomId={randomId}
bytes={bytes}
parsingConfig={parsingConfig}
xAxis={xAxis}
yAxis={yAxis}
lineLabel={lineLabel ?? yAxis}
title={title}
/>
</QueryClientProvider>
);
};
const LineChartInner: React.FC<PlotlyLineChartProps> = ({
url,
data,
rawCsv,
randomId,
bytes,
parsingConfig,
xAxis,
yAxis,
lineLabel,
title,
}) => {
if (data) {
return (
<div className="w-full" style={{ height: "500px" }}>
<Plotly
layout={{
title,
}}
data={[
{
x: data.map((d) => d[xAxis]),
y: data.map((d) => d[yAxis]),
mode: "lines",
name: lineLabel,
},
]}
/>
</div>
);
}
const { data: csvString, isLoading: isDownloadingCSV } = useQuery(
["dataCsv", url, randomId],
() => getCsv(url as string, bytes ?? 5132288),
{ enabled: !!url },
);
const { data: parsedData, isLoading: isParsing } = useQuery(
["dataPreview", csvString, randomId],
() =>
parseCsv(
rawCsv ? (rawCsv as string) : (csvString as string),
parsingConfig ?? {},
),
{ enabled: rawCsv ? true : !!csvString },
);
if (isParsing || isDownloadingCSV)
<div className="w-full flex justify-center items-center h-[500px]">
<LoadingSpinner />
</div>;
if (parsedData)
return (
<div className="w-full" style={{ height: "500px" }}>
<Plotly
layout={{
title,
}}
data={[
{
x: parsedData.data.map((d: any) => d[xAxis]),
y: parsedData.data.map((d: any) => d[yAxis]),
mode: "lines",
name: lineLabel,
},
]}
/>
</div>
);
return (
<div className="w-full flex justify-center items-center h-[500px]">
<LoadingSpinner />
</div>
);
};

View File

@@ -6,8 +6,6 @@ import {
getFilteredRowModel,
getPaginationRowModel,
getSortedRowModel,
PaginationState,
Table as ReactTable,
useReactTable,
} from '@tanstack/react-table';
@@ -27,19 +25,12 @@ import DebouncedInput from './DebouncedInput';
import loadData from '../lib/loadData';
import LoadingSpinner from './LoadingSpinner';
export type TableData = { cols: {key: string, name: string}[]; data: any[]; total: number };
export type TableProps = {
data?: Array<{ [key: string]: number | string }>;
cols?: Array<{ [key: string]: string }>;
csv?: string;
url?: string;
fullWidth?: boolean;
datastoreConfig?: {
dataStoreURI: string;
rowsPerPage?: number;
dataMapperFn: (data) => Promise<TableData> | TableData;
};
};
export const Table = ({
@@ -48,28 +39,8 @@ export const Table = ({
csv = '',
url = '',
fullWidth = false,
datastoreConfig,
}: TableProps) => {
const [isLoading, setIsLoading] = useState<boolean>(false);
const [pageMap, setPageMap] = useState(new Map<number, boolean>());
const {
dataMapperFn,
dataStoreURI,
rowsPerPage = 10,
} = datastoreConfig ?? {};
const [globalFilter, setGlobalFilter] = useState('');
const [isLoadingPage, setIsLoadingPage] = useState<boolean>(false);
const [totalOfRows, setTotalOfRows] = useState<number>(0);
const [{ pageIndex, pageSize }, setPagination] = useState<PaginationState>({
pageIndex: 0,
pageSize: rowsPerPage,
});
const [lastIndex, setLastIndex] = useState(pageSize);
const [startIndex, setStartIndex] = useState(0);
const [hasSorted, setHasSorted] = useState(false);
if (csv) {
const out = parseCsv(csv);
@@ -91,43 +62,9 @@ export const Table = ({
);
}, [data, cols]);
let table: ReactTable<unknown>;
const [globalFilter, setGlobalFilter] = useState('');
if (datastoreConfig) {
useEffect(() => {
setIsLoading(true);
fetch(`${dataStoreURI}&limit=${rowsPerPage}&offset=0`)
.then((res) => res.json())
.then(async (res) => {
const { data, cols, total } = await dataMapperFn(res);
setData(data);
setCols(cols);
setTotalOfRows(Math.ceil(total / rowsPerPage));
pageMap.set(0, true);
})
.finally(() => setIsLoading(false));
}, [dataStoreURI]);
table = useReactTable({
data,
pageCount: totalOfRows,
columns: tableCols,
getCoreRowModel: getCoreRowModel(),
state: {
pagination: { pageIndex, pageSize },
},
getFilteredRowModel: getFilteredRowModel(),
manualPagination: true,
onPaginationChange: setPagination,
getSortedRowModel: getSortedRowModel(),
});
useEffect(() => {
if (!hasSorted) return;
queryDataByText(globalFilter);
}, [table.getState().sorting]);
} else {
table = useReactTable({
const table = useReactTable({
data,
columns: tableCols,
getCoreRowModel: getCoreRowModel(),
@@ -140,7 +77,6 @@ export const Table = ({
getPaginationRowModel: getPaginationRowModel(),
getSortedRowModel: getSortedRowModel(),
});
}
useEffect(() => {
if (url) {
@@ -155,70 +91,6 @@ export const Table = ({
}
}, [url]);
const queryDataByText = (filter) => {
setIsLoadingPage(true);
const sortedParam = getSortParam();
fetch(
`${dataStoreURI}&limit=${rowsPerPage}&offset=0&q=${filter}${sortedParam}`
)
.then((res) => res.json())
.then(async (res) => {
const { data, total = 0 } = await dataMapperFn(res);
setTotalOfRows(Math.ceil(total / rowsPerPage));
setData(data);
const newMap = new Map();
newMap.set(0, true);
setPageMap(newMap);
table.setPageIndex(0);
setStartIndex(0);
setLastIndex(pageSize);
})
.finally(() => setIsLoadingPage(false));
};
const getSortParam = () => {
const sort = table.getState().sorting;
return sort.length == 0
? ``
: '&sort=' +
sort
.map(
(x, i) =>
`${x.id}${
i === sort.length - 1 ? (x.desc ? ` desc` : ` asc`) : `,`
}`
)
.reduce((x1, x2) => x1 + x2);
};
const queryPaginatedData = (newPageIndex) => {
let newStartIndex = newPageIndex * pageSize;
setStartIndex(newStartIndex);
setLastIndex(newStartIndex + pageSize);
if (!pageMap.get(newPageIndex)) pageMap.set(newPageIndex, true);
else return;
const sortedParam = getSortParam();
setIsLoadingPage(true);
fetch(
`${dataStoreURI}&limit=${rowsPerPage}&offset=${
newStartIndex + pageSize
}&q=${globalFilter}${sortedParam}`
)
.then((res) => res.json())
.then(async (res) => {
const { data: responseData } = await dataMapperFn(res);
responseData.forEach((e) => {
data[newStartIndex] = e;
newStartIndex++;
});
setData([...data]);
})
.finally(() => setIsLoadingPage(false));
};
return isLoading ? (
<div className="w-full h-full min-h-[500px] flex items-center justify-center">
<LoadingSpinner />
@@ -227,10 +99,7 @@ export const Table = ({
<div className={`${fullWidth ? 'w-[90vw] ml-[calc(50%-45vw)]' : 'w-full'}`}>
<DebouncedInput
value={globalFilter ?? ''}
onChange={(value: any) => {
if (datastoreConfig) queryDataByText(String(value));
setGlobalFilter(String(value));
}}
onChange={(value: any) => setGlobalFilter(String(value))}
className="p-2 text-sm shadow border border-block"
placeholder="Search all columns..."
/>
@@ -245,10 +114,7 @@ export const Table = ({
className: h.column.getCanSort()
? 'cursor-pointer select-none'
: '',
onClick: (v) => {
setHasSorted(true);
h.column.getToggleSortingHandler()(v);
},
onClick: h.column.getToggleSortingHandler(),
}}
>
{flexRender(h.column.columnDef.header, h.getContext())}
@@ -269,19 +135,7 @@ export const Table = ({
))}
</thead>
<tbody>
{datastoreConfig && isLoadingPage ? (
<tr>
<td colSpan={cols.length} rowSpan={cols.length}>
<div className="w-full h-full flex items-center justify-center pt-6">
<LoadingSpinner />
</div>
</td>
</tr>
) : (
(datastoreConfig
? table.getRowModel().rows.slice(startIndex, lastIndex)
: table.getRowModel().rows
).map((r) => (
{table.getRowModel().rows.map((r) => (
<tr key={r.id} className="border-b border-b-slate-200">
{r.getVisibleCells().map((c) => (
<td key={c.id} className="py-2">
@@ -289,8 +143,7 @@ export const Table = ({
</td>
))}
</tr>
))
)}
))}
</tbody>
</table>
<div className="flex gap-2 items-center justify-center mt-10">
@@ -298,10 +151,7 @@ export const Table = ({
className={`w-6 h-6 ${
!table.getCanPreviousPage() ? 'opacity-25' : 'opacity-100'
}`}
onClick={() => {
if (datastoreConfig) queryPaginatedData(0);
table.setPageIndex(0);
}}
onClick={() => table.setPageIndex(0)}
disabled={!table.getCanPreviousPage()}
>
<ChevronDoubleLeftIcon />
@@ -310,12 +160,7 @@ export const Table = ({
className={`w-6 h-6 ${
!table.getCanPreviousPage() ? 'opacity-25' : 'opacity-100'
}`}
onClick={() => {
if (datastoreConfig) {
queryPaginatedData(table.getState().pagination.pageIndex - 1);
}
table.previousPage();
}}
onClick={() => table.previousPage()}
disabled={!table.getCanPreviousPage()}
>
<ChevronLeftIcon />
@@ -331,11 +176,7 @@ export const Table = ({
className={`w-6 h-6 ${
!table.getCanNextPage() ? 'opacity-25' : 'opacity-100'
}`}
onClick={() => {
if (datastoreConfig)
queryPaginatedData(table.getState().pagination.pageIndex + 1);
table.nextPage();
}}
onClick={() => table.nextPage()}
disabled={!table.getCanNextPage()}
>
<ChevronRightIcon />
@@ -344,11 +185,7 @@ export const Table = ({
className={`w-6 h-6 ${
!table.getCanNextPage() ? 'opacity-25' : 'opacity-100'
}`}
onClick={() => {
const pageIndexToNavigate = table.getPageCount() - 1;
if (datastoreConfig) queryPaginatedData(pageIndexToNavigate);
table.setPageIndex(pageIndexToNavigate);
}}
onClick={() => table.setPageIndex(table.getPageCount() - 1)}
disabled={!table.getCanNextPage()}
>
<ChevronDoubleRightIcon />

View File

@@ -8,8 +8,3 @@ export * from './components/OpenLayers/OpenLayers';
export * from './components/Map';
export * from './components/PdfViewer';
export * from "./components/Excel";
export * from "./components/BucketViewer";
export * from "./components/Iframe";
export * from "./components/Plotly";
export * from "./components/PlotlyLineChart";
export * from "./components/PlotlyBarChart";

View File

@@ -1,74 +0,0 @@
import type { Meta, StoryObj } from '@storybook/react';
import { PlotlyBarChart, PlotlyBarChartProps } from '../src/components/PlotlyBarChart';
// More on how to set up stories at: https://storybook.js.org/docs/react/writing-stories/introduction
const meta: Meta = {
title: 'Components/PlotlyBarChart',
component: PlotlyBarChart,
tags: ['autodocs'],
argTypes: {
url: {
description:
'CSV Url to be parsed and used as data source',
},
data: {
description:
'Data to be displayed. as an array of key value pairs \n\n E.g.: [{ year: 1850, temperature: -0.41765878 }, { year: 1851, temperature: -0.2333498 }, ...]',
},
rawCsv: {
description:
'Raw csv data to be parsed and used as data source',
},
bytes: {
description:
'How many bytes to read from the url',
},
parsingConfig: {
description: 'If using url or rawCsv, this parsing config will be used to parse the data. Optional, check https://www.papaparse.com/ for more info',
},
title: {
description: 'Title to display on the chart. Optional.',
},
lineLabel: {
description: 'Label to display on the line, Optional, will use yAxis if not provided',
},
xAxis: {
description:
'Name of the X axis on the data. Required when the "data" parameter is an URL.',
},
yAxis: {
description:
'Name of the Y axis on the data. Required when the "data" parameter is an URL.',
},
},
};
export default meta;
type Story = StoryObj<PlotlyBarChartProps>;
export const FromDataPoints: Story = {
name: 'Line chart from array of data points',
args: {
data: [
{year: '1850', temperature: -0.41765878},
{year: '1851', temperature: -0.2333498},
{year: '1852', temperature: -0.22939907},
{year: '1853', temperature: -0.27035445},
{year: '1854', temperature: -0.29163003},
],
xAxis: 'year',
yAxis: 'temperature',
},
};
export const FromURL: Story = {
name: 'Line chart from URL',
args: {
title: 'Apple Stock Prices',
url: 'https://raw.githubusercontent.com/plotly/datasets/master/finance-charts-apple.csv',
xAxis: 'Date',
yAxis: 'AAPL.Open',
},
};

View File

@@ -1,97 +0,0 @@
import { type Meta, type StoryObj } from '@storybook/react';
import {
BucketViewer,
BucketViewerProps,
} from '../src/components/BucketViewer';
import LoadingSpinner from '../src/components/LoadingSpinner';
// More on how to set up stories at: https://storybook.js.org/docs/react/writing-stories/introduction
const meta: Meta = {
title: 'Components/BucketViewer',
component: BucketViewer,
tags: ['autodocs'],
argTypes: {
domain: {
description: 'Bucket domain URI',
},
suffix: {
description: 'Suffix of bucket domain',
},
downloadConfig: {
description: `Bucket file download configuration`,
},
filterState: {
description: `State with values used to filter the bucket files`,
},
paginationConfig: {
description: `Configuration to show and stylise the pagination on the component`,
},
},
};
export default meta;
type Story = StoryObj<BucketViewerProps>;
// More on writing stories with args: https://storybook.js.org/docs/react/writing-stories/args
export const Normal: Story = {
name: 'Bucket viewer',
args: {
domain: 'https://ssen-smart-meter.datopian.workers.dev',
suffix: '/',
dataMapperFn: async (rawData: Response) => {
const result = await rawData.json();
return result.objects.map((e) => ({
downloadFileUri: e.downloadLink,
fileName: e.key.replace(/^(\w+\/)/g, ''),
dateProps: {
date: new Date(e.uploaded),
dateFormatter: (date) => date.toLocaleDateString(),
},
}));
},
},
};
export const WithPagination: Story = {
name: 'With pagination',
args: {
domain: 'https://ssen-smart-meter.datopian.workers.dev',
suffix: '/',
paginationConfig: {
itemsPerPage: 3,
},
dataMapperFn: async (rawData: Response) => {
const result = await rawData.json();
return result.objects.map((e) => ({
downloadFileUri: e.downloadLink,
fileName: e.key.replace(/^(\w+\/)/g, ''),
dateProps: {
date: new Date(e.uploaded),
dateFormatter: (date) => date.toLocaleDateString(),
},
}));
},
},
};
export const WithComponentOnHoverOfEachBucketFile: Story = {
name: 'With component on hover of each bucket file',
args: {
domain: 'https://ssen-smart-meter.datopian.workers.dev',
suffix: '/',
downloadConfig: { hoverOfTheFileComponent: `HOVER COMPONENT` },
dataMapperFn: async (rawData: Response) => {
const result = await rawData.json();
return result.objects.map((e) => ({
downloadFileUri: e.downloadLink,
fileName: e.key.replace(/^(\w+\/)/g, ''),
dateProps: {
date: new Date(e.uploaded),
dateFormatter: (date) => date.toLocaleDateString(),
},
}));
},
},
};

View File

@@ -64,6 +64,6 @@ export const FromRawCSV: Story = {
export const FromURL: Story = {
name: 'Table from URL',
args: {
url: 'https://storage.openspending.org/alberta-budget/__os_imported__alberta_total.csv',
url: 'https://ckan-dev.sse.datopian.com/datastore/dump/601c9cf0-595e-46d8-88fc-d1ab2904e2db',
},
};

View File

@@ -1,31 +0,0 @@
import { type Meta, type StoryObj } from '@storybook/react';
import { Iframe, IframeProps } from '../src/components/Iframe';
const meta: Meta = {
title: 'Components/Iframe',
component: Iframe,
tags: ['autodocs'],
argTypes: {
url: {
description:
'Page to display inside of the component',
},
style: {
description:
'Style of the component',
},
},
};
export default meta;
type Story = StoryObj<IframeProps>;
export const Normal: Story = {
name: 'Iframe',
args: {
url: 'https://app.powerbi.com/view?r=eyJrIjoiYzBmN2Q2MzYtYzE3MS00ODkxLWE5OWMtZTQ2MjBlMDljMDk4IiwidCI6Ijk1M2IwZjgzLTFjZTYtNDVjMy04MmM5LTFkODQ3ZTM3MjMzOSIsImMiOjh9',
style: {width: `100%`, height: `100%`}
},
};

View File

@@ -1,74 +0,0 @@
import type { Meta, StoryObj } from '@storybook/react';
import { PlotlyLineChart, PlotlyLineChartProps } from '../src/components/PlotlyLineChart';
// More on how to set up stories at: https://storybook.js.org/docs/react/writing-stories/introduction
const meta: Meta = {
title: 'Components/PlotlyLineChart',
component: PlotlyLineChart,
tags: ['autodocs'],
argTypes: {
url: {
description:
'CSV Url to be parsed and used as data source',
},
data: {
description:
'Data to be displayed. as an array of key value pairs \n\n E.g.: [{ year: 1850, temperature: -0.41765878 }, { year: 1851, temperature: -0.2333498 }, ...]',
},
rawCsv: {
description:
'Raw csv data to be parsed and used as data source',
},
bytes: {
description:
'How many bytes to read from the url',
},
parsingConfig: {
description: 'If using url or rawCsv, this parsing config will be used to parse the data. Optional, check https://www.papaparse.com/ for more info',
},
title: {
description: 'Title to display on the chart. Optional.',
},
lineLabel: {
description: 'Label to display on the line, Optional, will use yAxis if not provided',
},
xAxis: {
description:
'Name of the X axis on the data. Required when the "data" parameter is an URL.',
},
yAxis: {
description:
'Name of the Y axis on the data. Required when the "data" parameter is an URL.',
},
},
};
export default meta;
type Story = StoryObj<PlotlyLineChartProps>;
export const FromDataPoints: Story = {
name: 'Line chart from array of data points',
args: {
data: [
{year: '1850', temperature: -0.41765878},
{year: '1851', temperature: -0.2333498},
{year: '1852', temperature: -0.22939907},
{year: '1853', temperature: -0.27035445},
{year: '1854', temperature: -0.29163003},
],
xAxis: 'year',
yAxis: 'temperature',
},
};
export const FromURL: Story = {
name: 'Line chart from URL',
args: {
title: 'Oil Price x Year',
url: 'https://raw.githubusercontent.com/datasets/oil-prices/main/data/wti-year.csv',
xAxis: 'Date',
yAxis: 'Price',
},
};

View File

@@ -21,12 +21,6 @@ const meta: Meta = {
zoom: {
description: 'Zoom level',
},
style: {
description: "Styles for the container"
},
autoZoomConfiguration: {
description: "Configuration to auto zoom in the specified layer data"
}
},
};
@@ -94,32 +88,4 @@ export const GeoJSONMultipleLayers: Story = {
center: { latitude: 45, longitude: 0 },
zoom: 2,
},
}
export const GeoJSONMultipleLayersWithAutoZoomInSpecifiedLayer: Story = {
name: 'GeoJSON polygons and points map with auto zoom in the points layer',
args: {
layers: [
{
data: 'https://opendata.arcgis.com/datasets/9c58741995174fbcb017cf46c8a42f4b_25.geojson',
name: 'Points',
tooltip: true,
},
{
data: 'https://d2ad6b4ur7yvpq.cloudfront.net/naturalearth-3.3.0/ne_10m_geography_marine_polys.geojson',
name: 'Polygons',
tooltip: true,
colorScale: {
starting: '#ff0000',
ending: '#00ff00',
},
},
],
title: 'Polygons and points',
center: { latitude: 45, longitude: 0 },
zoom: 2,
autoZoomConfiguration: {
layerName: 'Points'
}
},
};

View File

@@ -1,39 +0,0 @@
import type { Meta, StoryObj } from '@storybook/react';
import { Plotly } from '../src/components/Plotly';
// More on how to set up stories at: https://storybook.js.org/docs/react/writing-stories/introduction
const meta: Meta = {
title: 'Components/Plotly',
component: Plotly,
tags: ['autodocs'],
};
export default meta;
type Story = StoryObj<any>;
// More on writing stories with args: https://storybook.js.org/docs/react/writing-stories/args
export const Primary: Story = {
name: 'Chart built with Plotly',
args: {
data: [
{
x: [1, 2, 3],
y: [2, 6, 3],
type: 'scatter',
mode: 'lines+markers',
marker: { color: 'red' },
},
],
layout: {
title: 'Chart built with Plotly',
xaxis: {
title: 'x Axis',
},
yaxis: {
title: 'y Axis',
},
},
},
};

View File

@@ -9,22 +9,17 @@ const meta: Meta = {
tags: ['autodocs'],
argTypes: {
data: {
description:
'Data to be displayed in the table, must also set "cols" to work.',
description: "Data to be displayed in the table, must also set \"cols\" to work."
},
cols: {
description:
'Columns to be displayed in the table, must also set "data" to work.',
description: "Columns to be displayed in the table, must also set \"data\" to work."
},
csv: {
description: 'CSV data as string.',
description: "CSV data as string.",
},
url: {
description: 'Fetch the data from a CSV file remotely.',
},
datastoreConfig: {
description: `Configuration to use CKAN's datastore API extension integrated with the component`,
},
description: "Fetch the data from a CSV file remotely."
}
},
};
@@ -34,7 +29,7 @@ type Story = StoryObj<TableProps>;
// More on writing stories with args: https://storybook.js.org/docs/react/writing-stories/args
export const FromColumnsAndData: Story = {
name: 'Table from columns and data',
name: "Table from columns and data",
args: {
data: [
{ id: 1, lastName: 'Snow', firstName: 'Jon', age: 35 },
@@ -54,40 +49,21 @@ export const FromColumnsAndData: Story = {
},
};
export const WithDataStoreIntegration: Story = {
name: 'Table with datastore integration',
args: {
datastoreConfig: {
dataStoreURI: `https://www.civicdata.com/api/action/datastore_search?resource_id=46ec0807-31ff-497f-bfa0-f31c796cdee8`,
dataMapperFn: ({
result,
}: {
result: { fields: { id }[]; records: []; total: number };
}) => {
return {
data: result.records,
cols: result.fields.map((x) => ({ key: x.id, name: x.id })),
total: result.total,
};
},
},
},
};
export const FromRawCSV: Story = {
name: 'Table from raw CSV',
name: "Table from raw CSV",
args: {
csv: `
Year,Temp Anomaly
1850,-0.418
2020,0.923
`,
},
`
}
};
export const FromURL: Story = {
name: 'Table from URL',
name: "Table from URL",
args: {
url: 'https://raw.githubusercontent.com/datasets/finance-vix/main/data/vix-daily.csv',
},
url: "https://raw.githubusercontent.com/datasets/finance-vix/main/data/vix-daily.csv"
}
};

View File

@@ -1,6 +1,6 @@
{
"name": "@portaljs/core",
"version": "1.0.9",
"version": "1.0.8",
"description": "Core Portal.JS components, configs and utils.",
"repository": {
"type": "git",

View File

@@ -46,8 +46,8 @@ export const SiteToc: React.FC<Props> = ({ currentPath, nav }) => {
return (
<nav data-testid="lhs-sidebar" className="flex flex-col space-y-3 text-sm">
{sortNavGroupChildren(nav).map((n, index) => (
<NavComponent key={index} item={n} isActive={false} />
{sortNavGroupChildren(nav).map((n) => (
<NavComponent item={n} isActive={false} />
))}
</nav>
);
@@ -96,8 +96,8 @@ const NavComponent: React.FC<{
leaveTo="transform scale-95 opacity-0"
>
<Disclosure.Panel className="flex flex-col space-y-3 pl-5 mt-3">
{sortNavGroupChildren(item.children).map((subItem, index) => (
<NavComponent key={index} item={subItem} isActive={false} />
{sortNavGroupChildren(item.children).map((subItem) => (
<NavComponent item={subItem} isActive={false} />
))}
</Disclosure.Panel>
</Transition>

View File

@@ -1,36 +0,0 @@
import Script from 'next/script.js'
export interface GoogleAnalyticsProps {
googleAnalyticsId: string
}
export const GA = ({ googleAnalyticsId }: GoogleAnalyticsProps) => {
return (
<>
<Script
strategy="afterInteractive"
src={`https://www.googletagmanager.com/gtag/js?id=${googleAnalyticsId}`}
/>
<Script strategy="afterInteractive" id="ga-script">
{`
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', '${googleAnalyticsId}');
`}
</Script>
</>
)
}
// https://developers.google.com/analytics/devguides/collection/gtagjs/events
export const logEvent = (action, category, label, value) => {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
window.gtag?.('event', action, {
event_category: category,
event_label: label,
value: value,
})
}

View File

@@ -1,41 +0,0 @@
import Script from 'next/script.js'
export interface PlausibleProps {
plausibleDataDomain: string
dataApi?: string
src?: string
}
/**
* Plausible analytics component.
* To proxy the requests through your own domain, you can use the dataApi and src attribute.
* See [Plausible docs](https://plausible.io/docs/proxy/guides/nextjs#step-2-adjust-your-deployed-script)
* for more information.
*
*/
export const Plausible = ({
plausibleDataDomain,
dataApi = undefined,
src = 'https://plausible.io/js/plausible.js',
}: PlausibleProps) => {
return (
<>
<Script
strategy="lazyOnload"
data-domain={plausibleDataDomain}
data-api={dataApi}
src={src}
/>
<Script strategy="lazyOnload" id="plausible-script">
{`
window.plausible = window.plausible || function() { (window.plausible.q = window.plausible.q || []).push(arguments) }
`}
</Script>
</>
)
}
// https://plausible.io/docs/custom-event-goals
export const logEvent = (eventName, ...rest) => {
return window.plausible?.(eventName, ...rest)
}

View File

@@ -1,25 +0,0 @@
import Script from 'next/script.js'
export interface PosthogProps {
posthogProjectApiKey: string
apiHost?: string
}
/**
* Posthog analytics component.
* See [Posthog docs](https://posthog.com/docs/libraries/js#option-1-add-javascript-snippet-to-your-html-badgerecommendedbadge) for more information.
*
*/
export const Posthog = ({
posthogProjectApiKey,
apiHost = 'https://app.posthog.com',
}: PosthogProps) => {
return (
<Script strategy="lazyOnload" id="posthog-script">
{`
!function(t,e){var o,n,p,r;e.__SV||(window.posthog=e,e._i=[],e.init=function(i,s,a){function g(t,e){var o=e.split(".");2==o.length&&(t=t[o[0]],e=o[1]),t[e]=function(){t.push([e].concat(Array.prototype.slice.call(arguments,0)))}}(p=t.createElement("script")).type="text/javascript",p.async=!0,p.src=s.api_host+"/static/array.js",(r=t.getElementsByTagName("script")[0]).parentNode.insertBefore(p,r);var u=e;for(void 0!==a?u=e[a]=[]:a="posthog",u.people=u.people||[],u.toString=function(t){var e="posthog";return"posthog"!==a&&(e+="."+a),t||(e+=" (stub)"),e},u.people.toString=function(){return u.toString(1)+".people (stub)"},o="capture identify alias people.set people.set_once set_config register register_once unregister opt_out_capturing has_opted_out_capturing opt_in_capturing reset isFeatureEnabled onFeatureFlags".split(" "),n=0;n<o.length;n++)g(u,o[n]);e._i.push([i,s,a])},e.__SV=1)}(document,window.posthog||[]);
posthog.init('${posthogProjectApiKey}',{api_host:'${apiHost}'})
`}
</Script>
)
}

View File

@@ -1,29 +0,0 @@
import Script from 'next/script.js'
export interface SimpleAnalyticsProps {
src?: string
}
export const SimpleAnalytics = ({
src = 'https://scripts.simpleanalyticscdn.com/latest.js',
}: SimpleAnalyticsProps) => {
return (
<>
<Script strategy="lazyOnload" id="sa-script">
{`
window.sa_event=window.sa_event||function(){var a=[].slice.call(arguments);window.sa_event.q?window.sa_event.q.push(a):window.sa_event.q=[a]};
`}
</Script>
<Script strategy="lazyOnload" src={src} />
</>
)
}
// https://docs.simpleanalytics.com/events
export const logEvent = (eventName, callback) => {
if (callback) {
return window.sa_event?.(eventName, callback)
} else {
return window.sa_event?.(eventName)
}
}

View File

@@ -1,20 +0,0 @@
import Script from 'next/script.js'
export interface UmamiProps {
umamiWebsiteId: string
src?: string
}
export const Umami = ({
umamiWebsiteId,
src = 'https://analytics.umami.is/script.js',
}: UmamiProps) => {
return (
<Script
async
defer
data-website-id={umamiWebsiteId}
src={src} // Replace with your umami instance
/>
)
}

View File

@@ -1,82 +0,0 @@
/* eslint-disable @typescript-eslint/no-explicit-any */
import { GA, GoogleAnalyticsProps } from "./GoogleAnalytics";
import { Plausible, PlausibleProps } from "./Plausible";
import { SimpleAnalytics, SimpleAnalyticsProps } from "./SimpleAnalytics";
import { Umami, UmamiProps } from "./Umami";
import { Posthog, PosthogProps } from "./Posthog";
declare global {
interface Window {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
gtag?: (...args: any[]) => void;
plausible?: (...args: any[]) => void;
sa_event?: (...args: any[]) => void;
}
}
export interface AnalyticsConfig {
googleAnalytics?: GoogleAnalyticsProps;
plausibleAnalytics?: PlausibleProps;
umamiAnalytics?: UmamiProps;
posthogAnalytics?: PosthogProps;
simpleAnalytics?: SimpleAnalyticsProps;
}
/**
* @example
* const analytics: AnalyticsConfig = {
* plausibleDataDomain: '', // e.g. tailwind-nextjs-starter-blog.vercel.app
* simpleAnalytics: false, // true or false
* umamiWebsiteId: '', // e.g. 123e4567-e89b-12d3-a456-426614174000
* posthogProjectApiKey: '', // e.g. AhnJK8392ndPOav87as450xd
* googleAnalyticsId: '', // e.g. UA-000000-2 or G-XXXXXXX
* }
*/
export interface AnalyticsProps {
analyticsConfig: AnalyticsConfig;
}
const isProduction = true || process.env["NODE_ENV"] === "production";
/**
* Supports Plausible, Simple Analytics, Umami, Posthog or Google Analytics.
* All components default to the hosted service, but can be configured to use a self-hosted
* or proxied version of the script by providing the `src` / `apiHost` props.
*
* Note: If you want to use an analytics provider you have to add it to the
* content security policy in the `next.config.js` file.
* @param {AnalyticsProps} { analytics }
* @return {*}
*/
export const Analytics = ({ analyticsConfig }: AnalyticsProps) => {
return (
<>
{isProduction && analyticsConfig.plausibleAnalytics && (
<Plausible {...analyticsConfig.plausibleAnalytics} />
)}
{isProduction && analyticsConfig.simpleAnalytics && (
<SimpleAnalytics {...analyticsConfig.simpleAnalytics} />
)}
{isProduction && analyticsConfig.posthogAnalytics && (
<Posthog {...analyticsConfig.posthogAnalytics} />
)}
{isProduction && analyticsConfig.umamiAnalytics && (
<Umami {...analyticsConfig.umamiAnalytics} />
)}
{isProduction && analyticsConfig.googleAnalytics && (
<GA {...analyticsConfig.googleAnalytics} />
)}
</>
);
};
export { GA, Plausible, SimpleAnalytics, Umami, Posthog };
export type {
GoogleAnalyticsProps,
PlausibleProps,
UmamiProps,
PosthogProps,
SimpleAnalyticsProps,
};

View File

@@ -21,4 +21,3 @@ export { SiteToc, NavItem, NavGroup } from "./SiteToc";
export { Comments, CommentsConfig } from "./Comments";
export { AuthorConfig } from "./types";
export { Hero } from "./Hero";
export { Analytics, AnalyticsConfig } from "./analytics";

View File

@@ -7,8 +7,6 @@ export const pageview = ({
analyticsID: string;
}) => {
if (typeof window.gtag !== undefined) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
window.gtag("config", analyticsID, {
page_path: url,
});
@@ -18,8 +16,6 @@ export const pageview = ({
// https://developers.google.com/analytics/devguides/collection/gtagjs/events
export const event = ({ action, category, label, value }) => {
if (typeof window.gtag !== undefined) {
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
// @ts-ignore
window.gtag("event", action, {
event_category: category,
event_label: label,

View File

@@ -1,17 +1,5 @@
# @portaljs/remark-wiki-link
## 1.2.0
### Minor Changes
- [#1084](https://github.com/datopian/datahub/pull/1084) [`57952e08`](https://github.com/datopian/datahub/commit/57952e0817770138881e7492dc9f43e9910b56a8) Thanks [@mohamedsalem401](https://github.com/mohamedsalem401)! - Add image resize feature
## 1.1.2
### Patch Changes
- [#1040](https://github.com/datopian/portaljs/pull/1040) [`85bb6cb9`](https://github.com/datopian/portaljs/commit/85bb6cb98c53bedc2add3d014927570b5dd1bbdf) Thanks [@Gutts-n](https://github.com/Gutts-n)! - Changed regex to permit any symbols other than #
## 1.1.1
### Patch Changes

View File

@@ -1,6 +1,6 @@
{
"name": "@portaljs/remark-wiki-link",
"version": "1.2.0",
"version": "1.1.1",
"description": "Parse and render wiki-style links in markdown especially Obsidian style links.",
"repository": {
"type": "git",

View File

@@ -1,23 +1,23 @@
import { isSupportedFileFormat } from './isSupportedFileFormat';
import { isSupportedFileFormat } from "./isSupportedFileFormat";
const defaultWikiLinkResolver = (target: string) => {
// for [[#heading]] links
if (!target) {
return [];
}
let permalink = target.replace(/\/index$/, '');
let permalink = target.replace(/\/index$/, "");
// TODO what to do with [[index]] link?
if (permalink.length === 0) {
permalink = '/';
permalink = "/";
}
return [permalink];
};
export interface FromMarkdownOptions {
pathFormat?:
| 'raw' // default; use for regular relative or absolute paths
| 'obsidian-absolute' // use for Obsidian-style absolute paths (with no leading slash)
| 'obsidian-short'; // use for Obsidian-style shortened paths (shortest path possible)
| "raw" // default; use for regular relative or absolute paths
| "obsidian-absolute" // use for Obsidian-style absolute paths (with no leading slash)
| "obsidian-short"; // use for Obsidian-style shortened paths (shortest path possible)
permalinks?: string[]; // list of permalinks to match possible permalinks of a wiki link against
wikiLinkResolver?: (name: string) => string[]; // function to resolve wiki links to an array of possible permalinks
newClassName?: string; // class name to add to links that don't have a matching permalink
@@ -25,23 +25,14 @@ export interface FromMarkdownOptions {
hrefTemplate?: (permalink: string) => string; // function to generate the href attribute of a link
}
export function getImageSize(size: string) {
// eslint-disable-next-line prefer-const
let [width, height] = size.split('x');
if (!height) height = width;
return { width, height };
}
// mdas-util-from-markdown extension
// https://github.com/syntax-tree/mdast-util-from-markdown#extension
function fromMarkdown(opts: FromMarkdownOptions = {}) {
const pathFormat = opts.pathFormat || 'raw';
const pathFormat = opts.pathFormat || "raw";
const permalinks = opts.permalinks || [];
const wikiLinkResolver = opts.wikiLinkResolver || defaultWikiLinkResolver;
const newClassName = opts.newClassName || 'new';
const wikiLinkClassName = opts.wikiLinkClassName || 'internal';
const newClassName = opts.newClassName || "new";
const wikiLinkClassName = opts.wikiLinkClassName || "internal";
const defaultHrefTemplate = (permalink: string) => permalink;
const hrefTemplate = opts.hrefTemplate || defaultHrefTemplate;
@@ -53,9 +44,9 @@ function fromMarkdown(opts: FromMarkdownOptions = {}) {
function enterWikiLink(token) {
this.enter(
{
type: 'wikiLink',
type: "wikiLink",
data: {
isEmbed: token.isType === 'embed',
isEmbed: token.isType === "embed",
target: null, // the target of the link, e.g. "Foo Bar#Heading" in "[[Foo Bar#Heading]]"
alias: null, // the alias of the link, e.g. "Foo" in "[[Foo Bar|Foo]]"
permalink: null, // TODO shouldn't this be named just "link"?
@@ -88,19 +79,19 @@ function fromMarkdown(opts: FromMarkdownOptions = {}) {
data: { isEmbed, target, alias },
} = wikiLink;
// eslint-disable-next-line no-useless-escape
const wikiLinkWithHeadingPattern = /^(.*?)(#.*)?$/u;
const [, path, heading = ''] = target.match(wikiLinkWithHeadingPattern);
const wikiLinkWithHeadingPattern = /([\p{Letter}\d\s\/\.-_]*)(#.*)?/u;
const [, path, heading = ""] = target.match(wikiLinkWithHeadingPattern);
const possibleWikiLinkPermalinks = wikiLinkResolver(path);
const matchingPermalink = permalinks.find((e) => {
return possibleWikiLinkPermalinks.find((p) => {
if (pathFormat === 'obsidian-short') {
if (pathFormat === "obsidian-short") {
if (e === p || e.endsWith(p)) {
return true;
}
} else if (pathFormat === 'obsidian-absolute') {
if (e === '/' + p) {
} else if (pathFormat === "obsidian-absolute") {
if (e === "/" + p) {
return true;
}
} else {
@@ -115,19 +106,20 @@ function fromMarkdown(opts: FromMarkdownOptions = {}) {
// TODO this is ugly
const link =
matchingPermalink ||
(pathFormat === 'obsidian-absolute'
? '/' + possibleWikiLinkPermalinks[0]
(pathFormat === "obsidian-absolute"
? "/" + possibleWikiLinkPermalinks[0]
: possibleWikiLinkPermalinks[0]) ||
'';
"";
wikiLink.data.exists = !!matchingPermalink;
wikiLink.data.permalink = link;
// remove leading # if the target is a heading on the same page
const displayName = alias || target.replace(/^#/, '');
const headingId = heading.replace(/\s+/g, '-').toLowerCase();
const displayName = alias || target.replace(/^#/, "");
const headingId = heading.replace(/\s+/, "-").toLowerCase();
let classNames = wikiLinkClassName;
if (!matchingPermalink) {
classNames += ' ' + newClassName;
classNames += " " + newClassName;
}
if (isEmbed) {
@@ -135,55 +127,44 @@ function fromMarkdown(opts: FromMarkdownOptions = {}) {
if (!isSupportedFormat) {
// Temporarily render note transclusion as a regular wiki link
if (!format) {
wikiLink.data.hName = 'a';
wikiLink.data.hName = "a";
wikiLink.data.hProperties = {
className: classNames + ' ' + 'transclusion',
className: classNames + " " + "transclusion",
href: hrefTemplate(link) + headingId,
};
wikiLink.data.hChildren = [{ type: 'text', value: displayName }];
wikiLink.data.hChildren = [{ type: "text", value: displayName }];
} else {
wikiLink.data.hName = 'p';
wikiLink.data.hName = "p";
wikiLink.data.hChildren = [
{
type: 'text',
type: "text",
value: `![[${target}]]`,
},
];
}
} else if (format === 'pdf') {
wikiLink.data.hName = 'iframe';
} else if (format === "pdf") {
wikiLink.data.hName = "iframe";
wikiLink.data.hProperties = {
className: classNames,
width: '100%',
width: "100%",
src: `${hrefTemplate(link)}#toolbar=0`,
};
} else {
const hasDimensions = alias && /^\d+(x\d+)?$/.test(alias);
// Take the target as alt text except if alt name was provided [[target|alt text]]
const altText = hasDimensions || !alias ? target : alias;
wikiLink.data.hName = 'img';
wikiLink.data.hName = "img";
wikiLink.data.hProperties = {
className: classNames,
src: hrefTemplate(link),
alt: altText
alt: displayName,
};
if (hasDimensions) {
const { width, height } = getImageSize(alias as string);
Object.assign(wikiLink.data.hProperties, {
width,
height,
});
}
}
} else {
wikiLink.data.hName = 'a';
wikiLink.data.hName = "a";
wikiLink.data.hProperties = {
className: classNames,
href: hrefTemplate(link) + headingId,
};
wikiLink.data.hChildren = [{ type: 'text', value: displayName }];
wikiLink.data.hChildren = [{ type: "text", value: displayName }];
}
}

View File

@@ -1,24 +1,23 @@
import { getImageSize } from './fromMarkdown';
import { isSupportedFileFormat } from './isSupportedFileFormat';
import { isSupportedFileFormat } from "./isSupportedFileFormat";
const defaultWikiLinkResolver = (target: string) => {
// for [[#heading]] links
if (!target) {
return [];
}
let permalink = target.replace(/\/index$/, '');
let permalink = target.replace(/\/index$/, "");
// TODO what to do with [[index]] link?
if (permalink.length === 0) {
permalink = '/';
permalink = "/";
}
return [permalink];
};
export interface HtmlOptions {
pathFormat?:
| 'raw' // default; use for regular relative or absolute paths
| 'obsidian-absolute' // use for Obsidian-style absolute paths (with no leading slash)
| 'obsidian-short'; // use for Obsidian-style shortened paths (shortest path possible)
| "raw" // default; use for regular relative or absolute paths
| "obsidian-absolute" // use for Obsidian-style absolute paths (with no leading slash)
| "obsidian-short"; // use for Obsidian-style shortened paths (shortest path possible)
permalinks?: string[]; // list of permalinks to match possible permalinks of a wiki link against
wikiLinkResolver?: (name: string) => string[]; // function to resolve wiki links to an array of possible permalinks
newClassName?: string; // class name to add to links that don't have a matching permalink
@@ -29,11 +28,11 @@ export interface HtmlOptions {
// Micromark HtmlExtension
// https://github.com/micromark/micromark#htmlextension
function html(opts: HtmlOptions = {}) {
const pathFormat = opts.pathFormat || 'raw';
const pathFormat = opts.pathFormat || "raw";
const permalinks = opts.permalinks || [];
const wikiLinkResolver = opts.wikiLinkResolver || defaultWikiLinkResolver;
const newClassName = opts.newClassName || 'new';
const wikiLinkClassName = opts.wikiLinkClassName || 'internal';
const newClassName = opts.newClassName || "new";
const wikiLinkClassName = opts.wikiLinkClassName || "internal";
const defaultHrefTemplate = (permalink: string) => permalink;
const hrefTemplate = opts.hrefTemplate || defaultHrefTemplate;
@@ -42,21 +41,21 @@ function html(opts: HtmlOptions = {}) {
}
function enterWikiLink() {
let stack = this.getData('wikiLinkStack');
if (!stack) this.setData('wikiLinkStack', (stack = []));
let stack = this.getData("wikiLinkStack");
if (!stack) this.setData("wikiLinkStack", (stack = []));
stack.push({});
}
function exitWikiLinkTarget(token) {
const target = this.sliceSerialize(token);
const current = top(this.getData('wikiLinkStack'));
const current = top(this.getData("wikiLinkStack"));
current.target = target;
}
function exitWikiLinkAlias(token) {
const alias = this.sliceSerialize(token);
const current = top(this.getData('wikiLinkStack'));
const current = top(this.getData("wikiLinkStack"));
current.alias = alias;
}
@@ -65,7 +64,7 @@ function html(opts: HtmlOptions = {}) {
const { target, alias } = wikiLink;
const isEmbed = token.isType === "embed";
// eslint-disable-next-line no-useless-escape
const wikiLinkWithHeadingPattern = /^(.*?)(#.*)?$/u;
const wikiLinkWithHeadingPattern = /([\w\s\/\.-]*)(#.*)?/;
const [, path, heading = ""] = target.match(wikiLinkWithHeadingPattern);
const possibleWikiLinkPermalinks = wikiLinkResolver(path);
@@ -100,7 +99,7 @@ function html(opts: HtmlOptions = {}) {
// remove leading # if the target is a heading on the same page
const displayName = alias || target.replace(/^#/, "");
// replace spaces with dashes and lowercase headings
const headingId = heading.replace(/\s+/g, "-").toLowerCase();
const headingId = heading.replace(/\s+/, "-").toLowerCase();
let classNames = wikiLinkClassName;
if (!matchingPermalink) {
classNames += " " + newClassName;
@@ -112,9 +111,7 @@ function html(opts: HtmlOptions = {}) {
// Temporarily render note transclusion as a regular wiki link
if (!format) {
this.tag(
`<a href="${hrefTemplate(
link + headingId
)}" class="${classNames} transclusion">`
`<a href="${hrefTemplate(link + headingId)}" class="${classNames} transclusion">`
);
this.raw(displayName);
this.tag("</a>");
@@ -128,18 +125,11 @@ function html(opts: HtmlOptions = {}) {
)}#toolbar=0" class="${classNames}" />`
);
} else {
const hasDimensions = alias && /^\d+(x\d+)?$/.test(alias);
// Take the target as alt text except if alt name was provided [[target|alt text]]
const altText = hasDimensions || !alias ? target : alias;
let imgAttributes = `src="${hrefTemplate(
this.tag(
`<img src="${hrefTemplate(
link
)}" alt="${altText}" class="${classNames}"`;
if (hasDimensions) {
const { width, height } = getImageSize(alias as string);
imgAttributes += ` width="${width}" height="${height}"`;
}
this.tag(`<img ${imgAttributes} />`);
)}" alt="${displayName}" class="${classNames}" />`
);
}
} else {
this.tag(

View File

@@ -6,7 +6,7 @@ import { getPermalinks } from "../src/utils";
// const markdownFolder = path.join(__dirname, "/fixtures/content");
const markdownFolder = path.join(
".",
"test/fixtures/content"
"/packages/remark-wiki-link/test/fixtures/content"
);
describe("getPermalinks", () => {

View File

@@ -48,7 +48,7 @@ describe("micromark-extension-wiki-link", () => {
html({
permalinks: ["/some/folder/Wiki Link"],
pathFormat: "obsidian-short",
}) as any, // TODO type fix
}) as any // TODO type fix
],
});
expect(serialized).toBe(
@@ -75,7 +75,7 @@ describe("micromark-extension-wiki-link", () => {
html({
permalinks: ["/some/folder/Wiki Link"],
pathFormat: "obsidian-absolute",
}) as any, // TODO type fix
}) as any // TODO type fix
],
});
expect(serialized).toBe(
@@ -97,14 +97,10 @@ describe("micromark-extension-wiki-link", () => {
});
test("parses a wiki link with heading and alias", () => {
const serialized = micromark(
"[[Wiki Link#Some Heading|Alias]]",
"ascii",
{
const serialized = micromark("[[Wiki Link#Some Heading|Alias]]", "ascii", {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
}
);
});
// note: lowercased and hyphenated heading
expect(serialized).toBe(
'<p><a href="Wiki Link#some-heading" class="internal new">Alias</a></p>'
@@ -138,7 +134,7 @@ describe("micromark-extension-wiki-link", () => {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
});
expect(serialized).toBe('<p>![[My Image.xyz]]</p>');
expect(serialized).toBe("<p>![[My Image.xyz]]</p>");
});
test("parses and image ambed with a matching permalink", () => {
@@ -151,28 +147,6 @@ describe("micromark-extension-wiki-link", () => {
);
});
// TODO: Fix alt attribute
test("Can identify the dimensions of the image if exists", () => {
const serialized = micromark("![[My Image.jpg|200]]", "ascii", {
extensions: [syntax()],
htmlExtensions: [html({ permalinks: ["My Image.jpg"] }) as any], // TODO type fix
});
expect(serialized).toBe(
'<p><img src="My Image.jpg" alt="My Image.jpg" class="internal" width="200" height="200" /></p>'
);
});
// TODO: Fix alt attribute
test("Can identify the dimensions of the image if exists", () => {
const serialized = micromark("![[My Image.jpg|200x200]]", "ascii", {
extensions: [syntax()],
htmlExtensions: [html({ permalinks: ["My Image.jpg"] }) as any], // TODO type fix
});
expect(serialized).toBe(
'<p><img src="My Image.jpg" alt="My Image.jpg" class="internal" width="200" height="200" /></p>'
);
});
test("parses an image embed with a matching permalink and Obsidian-style shortedned path", () => {
const serialized = micromark("![[My Image.jpg]]", {
extensions: [syntax()],
@@ -180,7 +154,7 @@ describe("micromark-extension-wiki-link", () => {
html({
permalinks: ["/assets/My Image.jpg"],
pathFormat: "obsidian-short",
}) as any, // TODO type fix
}) as any // TODO type fix
],
});
expect(serialized).toBe(
@@ -215,7 +189,7 @@ describe("micromark-extension-wiki-link", () => {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
});
expect(serialized).toBe('<p>[[Wiki Link</p>');
expect(serialized).toBe("<p>[[Wiki Link</p>");
});
test("doesn't parse a wiki link with one missing closing bracket", () => {
@@ -223,7 +197,7 @@ describe("micromark-extension-wiki-link", () => {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
});
expect(serialized).toBe('<p>[[Wiki Link]</p>');
expect(serialized).toBe("<p>[[Wiki Link]</p>");
});
test("doesn't parse a wiki link with a missing opening bracket", () => {
@@ -231,7 +205,7 @@ describe("micromark-extension-wiki-link", () => {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
});
expect(serialized).toBe('<p>[Wiki Link]]</p>');
expect(serialized).toBe("<p>[Wiki Link]]</p>");
});
test("doesn't parse a wiki link in single brackets", () => {
@@ -239,7 +213,7 @@ describe("micromark-extension-wiki-link", () => {
extensions: [syntax()],
htmlExtensions: [html() as any], // TODO type fix
});
expect(serialized).toBe('<p>[Wiki Link]</p>');
expect(serialized).toBe("<p>[Wiki Link]</p>");
});
});
@@ -251,7 +225,7 @@ describe("micromark-extension-wiki-link", () => {
html({
newClassName: "test-new",
wikiLinkClassName: "test-wiki-link",
}) as any, // TODO type fix
}) as any // TODO type fix
],
});
expect(serialized).toBe(
@@ -277,7 +251,7 @@ describe("micromark-extension-wiki-link", () => {
wikiLinkResolver: (page) => [
page.replace(/\s+/, "-").toLowerCase(),
],
}) as any, // TODO type fix
}) as any // TODO type fix
],
});
expect(serialized).toBe(
@@ -347,14 +321,4 @@ describe("micromark-extension-wiki-link", () => {
);
});
});
describe("Links with special characters", () => {
test("parses a link with special characters and symbols", () => {
const serialized = micromark("[[li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#LI NK-W(i)th-àcèô íã_a(n)d_uNdErlinE!:ª%@'*º$ °~./\\]]", "ascii", {
extensions: [syntax()],
htmlExtensions: [html() as any],
});
expect(serialized).toBe(`<p><a href="li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#li-nk-w(i)th-àcèô-íã_a(n)d_underline!:ª%@'*º$-°~./\\" class="internal new">li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#LI NK-W(i)th-àcèô íã_a(n)d_uNdErlinE!:ª%@'*º$ °~./\\</a></p>`);
});
});
});

View File

@@ -246,28 +246,6 @@ describe("remark-wiki-link", () => {
expect(node.data?.hName).toEqual("img");
expect((node.data?.hProperties as any).src).toEqual("My Image.png");
expect((node.data?.hProperties as any).alt).toEqual("My Image.png");
expect((node.data?.hProperties as any).width).toBeUndefined();
expect((node.data?.hProperties as any).height).toBeUndefined();
});
});
test("Can identify the dimensions of the image if exists", () => {
const processor = unified().use(markdown).use(wikiLinkPlugin);
let ast = processor.parse("![[My Image.png|132x612]]");
ast = processor.runSync(ast);
expect(select("wikiLink", ast)).not.toEqual(null);
visit(ast, "wikiLink", (node: Node) => {
expect(node.data?.isEmbed).toEqual(true);
expect(node.data?.target).toEqual("My Image.png");
expect(node.data?.permalink).toEqual("My Image.png");
expect(node.data?.hName).toEqual("img");
expect((node.data?.hProperties as any).src).toEqual("My Image.png");
expect((node.data?.hProperties as any).alt).toEqual("My Image.png");
expect((node.data?.hProperties as any).width).toBe("132");
expect((node.data?.hProperties as any).height).toBe("612");
});
});
@@ -383,36 +361,6 @@ describe("remark-wiki-link", () => {
});
});
describe("Links with special characters", () => {
test("parses a link with special characters and symbols", () => {
const processor = unified().use(markdown).use(wikiLinkPlugin);
let ast = processor.parse(
"[[li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#li-nk-w(i)th-àcèô íã_a(n)D_UNDERLINE!:ª%@'*º$ °~./\\]]"
);
ast = processor.runSync(ast);
expect(select("wikiLink", ast)).not.toEqual(null);
visit(ast, "wikiLink", (node: Node) => {
expect(node.data?.exists).toEqual(false);
expect(node.data?.permalink).toEqual(
"li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\"
);
expect(node.data?.alias).toEqual(null);
expect(node.data?.hName).toEqual("a");
expect((node.data?.hProperties as any).className).toEqual(
"internal new"
);
expect((node.data?.hProperties as any).href).toEqual(
"li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#li-nk-w(i)th-àcèô-íã_a(n)d_underline!:ª%@'*º$-°~./\\"
);
expect((node.data?.hChildren as any)[0].value).toEqual(
"li nk-w(i)th-àcèô íã_a(n)d_underline!:ª%@'*º$ °~./\\#li-nk-w(i)th-àcèô íã_a(n)D_UNDERLINE!:ª%@'*º$ °~./\\"
);
});
});
});
describe("invalid wiki links", () => {
test("doesn't parse a wiki link with two missing closing brackets", () => {
const processor = unified().use(markdown).use(wikiLinkPlugin);

Binary file not shown.

Before

Width:  |  Height:  |  Size: 96 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 144 KiB

View File

@@ -1,17 +1,15 @@
---
title: 'Adding Maps to PortalJS: Enhancing Geospatial Data Visualization with PortalJS'
title: 'Enhancing Geospatial Data Visualization with PortalJS'
date: 2023-07-18
authors: ['João Demenech', 'Luccas Mateus', 'Yoana Popova']
filetype: 'blog'
---
This post walks you though adding maps and geospatial visualizations to PortalJS.
Are you keen on building rich and interactive data portals? Do you find value in the power and flexibility of JavaScript, Nextjs, and React? In that case, allow us to introduce you to [PortalJS](https://portaljs.org/), a state-of-the-art framework leveraging these technologies to help you build amazing data portals.
Are you interested in building rich and interactive data portals? Do you find value in the power and flexibility of JavaScript, Nextjs, and React? If so, [PortalJS](https://portaljs.org/) is for you. It's a state-of-the-art framework leveraging these technologies to help you build rich data portals.
Perhaps you already understand that the effective data visualization lies in the adept utilization of various data components. Within [PortalJS](https://portaljs.org/), we take data visualization a step further. It's not just about displaying data - it's about telling a captivating story through the strategic orchestration of a diverse array of data components.
Effective data visualization lies in the use of various data components. Within [PortalJS](https://portaljs.org/), we take data visualization a step further. It's not just about displaying data - it's about telling a story through combining a variety of data components.
In this post we will share our latest enhancement to PortalJS: maps, a powerful tool for visualizing geospatial data. In this post, we will to take you on a tour of our experiments and progress in enhancing map functionalities on PortalJS. The journey is still in its early stages, with new facets being unveiled and refined as we perfect our API.
We are now eager to share our latest enhancement to [PortalJS](https://portaljs.org/): maps, a powerful tool for visualizing geospatial data. In this post, we will to take you on a tour of our experiments and progress in enhancing map functionalities on [PortalJS](https://portaljs.org/). Our journey into this innovative feature is still in its early stages, with new facets being unveiled and refined as we perfect our API. Still, this exciting development opens a new avenue for visualizing data, enhancing your ability to convey complex geospatial information with clarity and precision.
## Exploring Map Formats

View File

@@ -3,7 +3,6 @@ title: 'Announcing MarkdownDB: an open source tool to create an SQL API to your
description: MarkdownDB - an open source library to transform markdown content into sql-queryable data. Build rich markdown-powered sites easily and reliably. New dedicated website at markdowndb.com
date: 2023-10-11
authors: ['Ola Rubaj']
filetype: blog
---
Hello, dear readers!

View File

@@ -2,7 +2,6 @@
title: What We Shipped in Jul-Aug 2023
authors: ['ola-rubaj']
date: 2023-09-2
filetype: blog
---
Hey everyone! 👋 Summer has been in full swing, and while I've managed to catch some vacation vibes, I've also been deep into code. I'm super excited to share some of the latest updates and features we've rolled out over the past two months. Let's dive in:

View File

@@ -1,34 +0,0 @@
---
title: 'The OpenSpending Revamp: Behind the Scenes'
date: 2023-10-13
authors: ['Luccas Mateus', 'João Demenech']
filetype: 'blog'
---
_This post was originally published on [the Datopian blog](http://datopian.com/blog/the-open-spending-revamp-behind-the-scenes)._
In our last article, we explored [the Open Spending revamp](https://www.datopian.com/blog/the-open-spending-revamp). Now, let's dive into the tech stack that makes it tick. We'll unpack how PortalJS, Cloudflare R2, Frictionless Data Packages, and Octokit come together to power this next-level data portal. From our Javascript framework PortalJS, that shapes the user experience, to Cloudflare R2, the robust storage solution that secures the data, we'll examine how each piece of technology contributes to the bigger picture. We'll also delve into the roles of Frictionless Data Packages for metadata management and Octokit for automating dataset metadata retrieval. Read on for the inside scoop!
## The Core: PortalJS
At the core of the revamped OpenSpending website is [PortalJS](https://portaljs.org), a JavaScript library that's a game-changer in building powerful data portals with data visualizations. What makes it so special? Well, it's packed with reusable React components that make our lives - and yours - a whole lot easier. Take, for example, our sleek CSV previews; they're brought to life by PortalJS' [FlatUI Component](https://storybook.portaljs.org/?path=/story/components-flatuitable--from-url). It helps transform raw numbers into visuals that you can easily understand and use. Curious to know more? Check out the [official PortalJS website](https://portaljs.org).
![Data visualization](/assets/blog/2023-10-13-the-open-spending-revamp-behind-the-scenes/data-visualization.png)
## Metadata: Frictionless Data Packages
Storing metadata might seem like a backstage operation, but it is pivotal. We chose Frictionless Data Packages, housed in the `os-data` GitHub organization as repositories, to serve this purpose. Frictionless Data Packages offer a simple but powerful format for cataloging and packaging a collection of data - in our scenario, that's primarily tabular data. These aren't merely storage bins - they align with FAIR principles, ensuring that the data is easily Findable, Accessible, Interoperable, and Reusable. This alignment positions them as an ideal solution for publishing datasets designed to be both openly accessible and highly usable. Learn more from their [official documentation](https://framework.frictionlessdata.io/).
## The Link: Octokit
Can you imagine having to manually gather metadata for each dataset from multiple GitHub repositories? Sounds tedious, right? Thats why we used Octokit, a GitHub API client for Node.js. This tool takes care of the heavy lifting, automating the metadata retrieval process for us. If you're intrigued by Octokit's capabilities, you can discover more in its [GitHub repository](https://github.com/octokit/octokit.js). To explore the datasets we've been working on, take a look at [OpenSpending Datasets](https://github.com/os-data).
## Storage: Cloudflare R2
When it comes to data storage, Cloudflare R2 emerges as our choice, defined by its blend of speed and reliability. This service empowers developers to securely store large amounts of blob data without the costly egress bandwidth fees associated with typical cloud storage services. For a comprehensive understanding of Cloudflare R2, their [blog post](https://cloudflare.net/news/news-details/2021/Cloudflare-Announces-R2-Storage-Rapid-and-Reliable-S3-Compatible-Object-Storage-Designed-for-the-Edge/default.aspx) serves as an excellent resource.
## In Closing
In closing, we invite you to explore the architecture and code that power this project. It's all openly accessible in our [GitHub repository](https://github.com/datopian/portaljs/tree/main/examples/openspending). Should you want to experience the end result firsthand, feel free to visit [openspending.org](https://www.openspending.org/). If you encounter any issues or have suggestions to improve the project, we welcome your contributions via our [GitHub issues page](https://github.com/datopian/portaljs/issues). For real-time assistance and to engage with our community, don't hesitate to join our [Discord Channel](https://discord.com/invite/EeyfGrGu4U). Thank you for taking the time to read about our work! We look forward to fostering a collaborative environment where knowledge is freely shared and continually enriched. ♥️
![FlatUiTable Code Snippet](/assets/blog/2023-10-13-the-open-spending-revamp-behind-the-scenes/code-example.png)

View File

@@ -23,7 +23,8 @@ const config = {
{ name: 'Guide', href: '/guide' },
{
name: 'Examples',
href: '/examples/'
href: 'https://github.com/datopian/portaljs/tree/main/examples',
target: '_blank',
},
{
name: 'Components',
@@ -69,7 +70,7 @@ const config = {
},
},
github: 'https://github.com/datopian/portaljs',
discord: 'https://discord.gg/xfFDMPU9dC',
discord: 'https://discord.gg/EeyfGrGu4U',
tableOfContents: true,
analytics: 'G-96GWZHMH57',
// editLinkShow: true,

View File

@@ -1,5 +0,0 @@
# Examples
For now, see the examples folder in github:
https://github.com/datopian/portaljs/tree/main/examples

View File

12
tools/tsconfig.tools.json Normal file
View File

@@ -0,0 +1,12 @@
{
"extends": "../tsconfig.base.json",
"compilerOptions": {
"outDir": "../dist/out-tsc/tools",
"rootDir": ".",
"module": "commonjs",
"target": "es5",
"types": ["node"],
"importHelpers": false
},
"include": ["**/*.ts"]
}