[examples/openspending] - openspending v0.2 (#907)

* [examples/openspending] - openspending v0.2

* [examples/openspending][m] - fix build

* [examples/openspending][xs] - fix build

* [examples/openspending][xs] - add prebuild step

* [examples/openspending][m] - fix requested by demenech

* [examples/openspending][sm] - remove links + fix bug

examples/openspending/content/help/aggregate.md

@@ -0,0 +1,153 @@
+---
+section: help
+lead: true
+title: Aggregate API
+authors:
+- Neil Ashton
+---
+The data source used to drive visualizations is the Aggregate API. It
+can be used to flexibly generate aggregated views of the data by
+applying filters and grouping criteria.
+
+This API is heavily based on OLAP concepts, and the documentation assumes
+you know [how we store data](../../help/guide/en/data-model/).
+
+#### Basic call and parameters
+
+    GET /api/2/aggregate?dataset=<dataset>
+
+Calls will return aggregation results as JSON. If no arguments other than the
+dataset are given, the whole cube is aggregated. The following parameters are supported:
+
+* ``dataset`` (required)
+  The dataset name to query.
+
+* ``measure``
+  The name of the measure over which aggregation will be performed. Defaults to
+  ``amount``.
+
+  Multiple measures in a single query are supported, separated by a pipe character:
+  ``measure=amount|budget`` (sums up the amount measure *and* the budget measure).
+
+* ``cut``
+  Filter the entries to use only a part of the cube. Only cells matching all the
+  criteria given will be used. With ``cut=time.year:2009``, you can filter for an
+  attribute value.
+
+  Multiple filters can be given separated by a pipe character:
+  ``cut=time.year:2009|category.name:health``. If two different filters are applied
+  to the same attribute, the query will include both results:
+  ``cut=time.year:2009|time.year:2010`` The dimensions you use for cut will be part
+  of the returned result.
+
+* ``drilldown``
+  Dimension to be drilled down to. Each drilldown will split the result set to create
+  a distinct result (cell) for each value of the dimension or attribute in
+  ``drilldown``.
+
+  For example, ``drilldown=time.year`` will return all entries in the source data
+  broken down by year. Multiple drilldowns can be combined: ``drilldown=year|category``
+  will return one cell for each year/category combination.
+
+* ``page``
+  Page number for paginated results. Defaults to ``1``.
+
+* ``pagesize``
+  Size of a page for paginated results. Defaults to ``10000``.
+
+* ``order``
+  List of attributes to be ordered by as a combination of ``criterion:dir``
+  pairs. The indicated direction is either ``asc`` for ascending order
+  or ``desc`` for descending order. For example, ``order=year:asc|category:asc``
+  sorts by year and then by category name.
+
+The API itself is inspired by [DataBrewery Cubes](http://packages.python.org/cubes/server.html#api),
+with which we aim to be compatible. At the moment, we only implement the ``aggregate`` call of
+this API and do not support hierarchical dimension queries in the same way.
+
+#### Result format
+
+The result will contain two keys, ``summary`` and ``drilldown``. The ``summary``
+represents an aggregation of the whole cuboid specified in the cut. The
+amount given is the sum of all drilldowns.
+
+The ``drilldown`` contains a cell for each value of each drilled-down
+dimension. Cells include the values of any attributes or dimensions
+which served as drilldown criteria, as well as the ``cut`` attributes.
+
+    {
+      "drilldown": [
+        {
+          "volume": {
+            "name": "section-i",
+            "label": "PARLIAMENT"
+          },
+          "amount": 267770600.0,
+          "num_entries": 46
+        },
+        {
+          "volume": {
+            "color": "#FF8C00",
+            "name": "section-ii",
+            "label": "COUNCIL"
+          },
+          "amount": 705435934.0,
+          "num_entries": 26
+        },
+      ],
+      "summary": {
+        "amount": 973206534.0,
+        "num_drilldowns": 2,
+        "num_entries": 72
+      }
+    }
+
+JSON is the default format but results of the aggregation can also be downloaded as a csv file. Just add ``format=csv`` to the URL parameters to fetch them as a csv file.
+
+#### Example: Where Does My Money Go?
+
+To highlight the use of this API, let's look at the UK Country
+Regional Analysis dataset. This is a high-level survey of the
+UK budget, and the original [Where Does My Money Go?](http://wheredoesmymoneygo.org)
+page was based on this data.
+
+The first call we'll make will aggregate the complete dataset
+and give us a total sum ([result](http://openspending.org/api/2/aggregate?dataset=ukgov-finances-cra)):
+
+    GET /api/2/aggregate?dataset=ukgov-finances-cra
+
+This is not very useful, however, as it includes UK spending
+over several years. So let's refine our query to include only
+2010 figures ([result](http://openspending.org/api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010)):
+
+    GET /api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010
+
+Much better! Now we may want to know how these funds are distributed
+geographically, so let's drill down by the [NUTS](http://epp.eurostat.ec.europa.eu/portal/page/portal/nuts_nomenclature/introduction)
+names of each region of the UK ([result](http://openspending.org/api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010&drilldown=region)):
+
+    GET /api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010&drilldown=region
+
+Given an SVG file with the right region names, this could easily be
+used to drive a CSS-based choropleth map, with a bit of JavaScript
+glue on the client side.
+
+Another set of dimensions of the CRA dataset is the [Classification of
+Functions of Government (COFOG)](http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=4),
+which classifies government activity by its functional purpose. Like
+many taxonomies, COFOG has several levels, which we have modelled as
+three dimensions: cofog1, cofog2 and cofog3.
+
+In order to generate a [BubbleTree](http://vis4.net/blog/posts/tutorial-bubble-tree/)
+diagram, we want to break down the full CRA dataset by each of these
+dimensions ([result](http://openspending.org/api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010&drilldown=cofog1|cofog2|cofog3)):
+
+    GET /api/2/aggregate?dataset=ukgov-finances-cra&cut=time.year:2010&drilldown=cofog1|cofog2|cofog3
+
+(Warning: this generates quite a lot of data. You may want to paginate
+the results to view it in your browser.)
+
+As you can see, the aggregator API can be used to flexibly query the
+data to generate views such as visualizations, maps or pivot tables.
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/conventions.md

@@ -0,0 +1,48 @@
+---
+section: help
+lead: true
+title: API conventions
+authors:
+- Neil Ashton
+redirect_from:
+- /help/api/
+---
+#### Authentication
+
+Some actions in OpenSpending require authentication, particularly those that write to the system or aim to access protected data (e.g. pre-publication datasets). For this purpose, each user is provided an API key. The key is displayed in the *settings* (go to the dashboard and click on *Change* next to the Information header). You can use it to perform authentication by adding the following into the HTTP headers (change <your-api-key> to the API key you find in your settings):
+
+    Authorization: ApiKey <your-api-key>
+
+#### JSON-P Callbacks
+
+All API calls that return JSON support JSON-P (JSON with padding). You can
+add a ``?callback=foo`` parameter to any query to wrap the output in a
+function call. This is used to include JSON data in other sites that do not
+support CORS:
+
+    $ curl http://openspending.org/cra.json?callback=foo
+
+    foo({
+        "description": "Data published by HM Treasury.",
+        "name": "cra",
+        "label": "Country Regional Analysis v2009",
+        "currency": "GBP"
+    });
+
+This can be used in remote web pages to include data as a simple ``script``
+tag:
+
+    <script>
+      function foo(data) {
+        alert(data.label);
+      }
+    </script>
+    <script src="http://openspending.org/cra.json?callback=foo"></script>
+
+<!--
+#### Cross Origin Resource Sharing
+
+The API does not yet support [CORS](http://code.google.com/p/html5security/wiki/CrossOriginRequestSecurity), but support will be added in the near future.
+-->
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/guide/adding-data-overview/index.md

@@ -0,0 +1,28 @@
+---
+section: help
+lead: true
+title: 'Adding data: overview'
+authors:
+- Neil Ashton
+---
+One of the most valuable contributions you can make to the OpenSpending project is to add a new dataset. This section of the guide walks you through the process of adding new data.
+
+A typical workflow for importing a dataset into OpenSpending involves the following steps:
+
+* Gather machine-readable data from a trustworthy source.
+
+* Convert the data to a CSV file in the format expected by OpenSpending, cleaning it to remove inconsistencies and errors.
+
+* Publish the data to the web.
+
+* Create a dataset add the published data as a new data source.
+
+* Model the dataset to assign a logical role to each column in the source table.
+
+* Load the data, or refine the data based on the feedback given by the platform about the data's consistency.
+
+Each of these steps will be explained in detail in the following sections.
+
+**Next**: [Gathering data](../gathering-data/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/create-viz/index.md

@@ -0,0 +1,50 @@
+---
+section: help
+lead: true
+title: Create a Visualization
+authors:
+- Neil Ashton
+---
+The OpenSpending platform makes it easy to create and embed visualizations of datasets. Three types of visualizations are supported: BubbleTree, TreeMap, and Table of Aggregates.
+All OpenSpending visualizations allow you to choose a series of dimensions along which to aggregate your data, drilling down into finer and finer detail. Each visualization is created the same way: by choosing the dimensions to aggregate and the order in which to drill down.
+To start creating a visualization, go to a dataset's home page and select **Create a visualization** from the *Visualizations* menu.
+
+#### BubbleTree
+
+The BubbleTree is an interactive visualization that presents aggregated spending data as a circle of bubbles. Each bubble represents an aggregated (sub-)total. The central bubble represents an aggregated sum, and its surrounding bubbles represent the other sums that it is made of. By clicking on any bubble, the user is shown how the sum breaks down into further sub-totals.
+
+To create a BubbleTree, choose the dimensions to aggregate and the order in which to aggregate them. Choose the primary dimension from the *Level* drop-down menu. You will see the aggregated total of that dimension as the central bubble, with values of the dimension surrounding it with their own totals.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_14.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_14.png" alt="image_14" width="478" height="553" class="alignnone size-full wp-image-598" /></a>
+
+To add a second level, click **Add a level** and choose a new dimension. Users will now be able to click on bubbles to "drill down" and see how the values of the first level break down into values on the second level.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_15.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_15.png" alt="image_15" width="473" height="564" class="alignnone size-full wp-image-599" /></a>
+
+#### TreeMap
+
+The TreeMap presents aggregated spending data as an interactive rectangle of coloured tiles. Each tile represents aggregated values for a particular dimension of the data. Clicking on the tile "zooms in" to show how it breaks down along further aggregated dimensions.
+
+To create a TreeMap, simply choose the dimensions to aggregate and their order. Select the primary dimension from the *Tile* menu. You will see a TreeMap showing how the total spending breaks down across that dimension.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_16.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_16.png" alt="image_16" width="475" height="543" class="alignnone size-full wp-image-600" /></a>
+
+The visualization has no useful interactivity yet. Adding further tile levels allows you to drill down to see how aggregated values decompose into smaller aggregates. To add a second level of tiles, click **Add a level** and choose a new dimension. Users can now click tiles to see how their totals break down.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_17.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_17.png" alt="image_17" width="475" height="561" class="alignnone size-full wp-image-601" /></a>
+
+#### Table of Aggregates
+
+The Table of Aggregates is a simple tabular view of a dataset that aggregates totals across chosen dimensions. A Table of Aggregates is specified by choosing dimensions for its columns.
+
+Choosing a primary dimension via the *Columns* menu will display the data in tabular form, with aggregated amounts and percentages of the overall total. By default, the rows will be sorted by percentage.
+
+<a href="http://blog.openspending.org/files/2013/08/image_18-e1375889043439.png"><img src="http://blog.openspending.org/files/2013/08/image_18-e1375889043439.png" alt="image_18" width="600" height="458" class="alignnone size-full wp-image-602" /></a>
+
+Adding another column by clicking **Add a level** will break down each subtotal in the first column by the aggregated sums of the new column. Note that this generally changes the percentage values and thus rearranges rows.
+
+<a href="http://blog.openspending.org/files/2013/08/image_19-e1375889063736.png"><img src="http://blog.openspending.org/files/2013/08/image_19-e1375889063736.png" alt="image_19" width="600" height="530" class="alignnone size-full wp-image-603" /></a>
+
+**Next**: [Embed a visualisation into your website](../embed-viz)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/creating-dataset/index.md

@@ -0,0 +1,54 @@
+---
+section: help
+lead: true
+title: Creating a dataset on OpenSpending
+authors:
+- Neil Ashton
+---
+To begin sharing data on the OpenSpending platform, register on OpenSpending.org and create a new OpenSpending dataset. To create a dataset, simply fill in some metadata that characterizes your data and provide the URL where your data is hosted.
+
+#### Creating a new dataset
+
+Log in to OpenSpending.org with your user information, or register if you have not yet done so. You will arrive at the Dashboard, where you will see a blue button labeled **Import a Dataset**. Click this to begin creating a new OpenSpending dataset.
+
+The next screen prompts you to provide metadata that characterizes your data. This includes the following fields:
+
+* *Title*: a descriptive and meaningful name for the dataset. Can be any string.
+
+* *Identifier*: a shorter title, used as part of the dataset's URL. Can only contain alphanumeric characters, dashes, and underscores – no whitespace or punctuation.
+
+* *Category*: one of "Budget", "Expenditure", and "Other". See the guide section on types of financial data for details on these categories.
+
+* *Currency*: the currency in which the spending described by the dataset takes place.
+
+* *Countries*: a list of countries referenced in the dataset. Choice of countries is constrained by a list of valid countries.
+
+* *Languages*: a list of languages used in the dataset. Choice of languages is constrained by a list of valid languages.
+
+* *Description*: a characterization of the dataset in simple prose. Can be any string.
+
+Fill in all of these fields. Be sure to include a Description which explains the origin of your dataset and acknowledges any changes you have introduced (for example, any cleaning you have done).
+
+Once all metadata has been filled in, press **Next Step** to proceed.
+
+#### Adding a new data source
+
+Clicking through to the next step creates your new OpenSpending dataset and takes you to its *Manage* page. The Manage page is used to add data sources. It is also used to provide schematic information that allows OpenSpending to interpret the data, a process called "modelling" that will be covered in the next section of the guide.
+
+To add a data source to a dataset, click **Add a source**. A prompt will appear, asking you for a URL. Provide the URL of the CSV file you published on the web in the previous section of the guide and click **Create**. You will see a blue text box indicating that OpenSpending is thinking about your data.
+
+<a href="http://blog.openspending.org/files/2013/08/image_2-e1375888360807.png"><img src="http://blog.openspending.org/files/2013/08/image_2-e1375888360807.png" alt="image_2" width="600" height="228" class="alignnone size-full wp-image-582" /></a>
+
+Click **Refresh** or simply use your browser's refresh button. If OpenSpending succeeded at analyzing your data, you should see a green text box telling you that your data is ready. You should also see a correct list of your CSV's columns.
+
+<a href="http://blog.openspending.org/files/2013/08/image_3-e1375888381459.png"><img src="http://blog.openspending.org/files/2013/08/image_3-e1375888381459.png" alt="image_3" width="600" height="408" class="alignnone size-full wp-image-583" /></a>
+
+Note that if you incorrectly provide OpenSpending with an HTML file instead of a valid CSV file, it will not complain but will simply try to analyze the HTML as if it were a CSV. The result looks like the following.
+
+<a href="http://blog.openspending.org/files/2013/08/image_4-e1375888407751.png"><img src="http://blog.openspending.org/files/2013/08/image_4-e1375888407751.png" alt="image_4" width="600" height="234" class="alignnone size-full wp-image-584" /></a>
+
+If you added a bad data source, don't worry. You do not have to use the source in your final dataset: OpenSpending requires you to do more work on a data source before it can be published. Simply add a new, correct source and forget about the bad one.
+
+**Next**: [Modelling your data in OpenSpending](../modelling-data/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/data-model/index.md

@@ -0,0 +1,28 @@
+---
+section: help
+lead: true
+title: How does OpenSpending represent data?
+authors:
+- Neil Ashton
+---
+OpenSpending maintains a collection of datasets, each of which represents a set of data from a separate source. Inside each dataset, individual transactions are represented by a set of entries. Each dataset has its own model that maps the structure of the data. The model encodes the properties of each dataset entry in terms of *dimensions*.
+
+#### Datasets
+
+The basic unit in the OpenSpending system is the dataset. Financial transactions sharing a common theme (e.g. a particular city’s spending, a budget for a particular year) are grouped together and stored as a dataset. A dataset is a collection of "entries", and each entry represents a single transaction associated with a quantity of money and a time.
+
+Datasets also include metadata to characterize their contents. The metadata includes a description of the dataset, information about the source of the data, and other such information which helps users find the dataset and interpret its contents.
+
+#### Models
+
+The structure of each dataset is completely up to the creator of the dataset. This structure is created by specifying a *model*, which provides the dimensions along which entries can differ from one another.
+
+A model consists of a set of *dimensions*. A dimension is a property that potentially differentiates one entry from another. If you imagine a dataset as a spreadsheet, each dimension can be thought of as a column. Dimensions can have more structure than an ordinary spreadsheet column, however.
+
+Dimensions come in several types. The most important is the *measure* type. Measures are dimensions which can contain a single numerical value. Another important dimension type is the *time* type, which represent dates and times. Every data needs at least one each of measure and time dimensions, representing respectively the amount of money represented by the transaction and the time when it took place.
+
+The remaining dimension types are used to represent other properties that entries might have, e.g. transaction numbers, labels from a classification scheme, or the names of individuals or companies involved. Such dimensions include *attributes*, which can hold a single value, and *compound dimensions*, which can hold a nested set of values. Compound dimensions are useful when a property includes several sub-properties which could each be used to aggregate the data.
+
+**Next**: [Adding data: overview](../adding-data-overview/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/embed-viz/index.md

@@ -0,0 +1,16 @@
+---
+section: help
+lead: true
+title: Embed a visualization into your website
+authors:
+- Neil Ashton
+---
+You can easily embed any of the visualizations created on OpenSpending on your own website. This means you can have the full interactive displays on your site.
+
+Let's say you have chosen a visualization on the OpenSpending platform. Notice there's an **Embed** button at the bottom right of the webpage. Click this button and you'll be presented with the code to embed the visualization on your website and some options for the size (in pixels) of the interactive. The rest is just cutting and pasting this code into your site. If you are unsure how to paste the code correctly, contact your site administrator.
+
+The reason it's possible to embed code comes down to *widgets*. In very simplified terms, a widget is a piece of code you can add into your webpage, and it pulls data – in this case, from the OpenSpending database – so you don't need to store datasets yourself.
+
+**Back to top**: [OpenSpending Guide](../)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/financial-data-types/index.md

@@ -0,0 +1,48 @@
+---
+section: help
+lead: true
+title: What types of financial data can go into OpenSpending?
+authors:
+- Neil Ashton
+---
+OpenSpending is very flexible in the types of financial data it supports. Although the OpenSpending project has a strong focus on government finance, this is not a technical constraint. OpenSpending supports any dataset consisting of a set of transactions, each associated with a quantity of money and a time.
+
+Most of the data currently hosted on OpenSpending can be categorized as either transactional or budgetary data. The main difference between these is their level of granularity. Transactional data tracks individual transactions, whereas budgetary data aggregates transactions into categories.
+
+#### Transactional spending data
+
+Transactional data, or simply "spending data", tracks individual financial transactions. Each payment from one entity to another on a given date and for a specific purpose (e.g. a project or service) is listed individually. Transactional spending data includes various types of records, including information on government grants, commitments, and actual expenditure.
+
+Aggregate information (e.g. subtotals) should not be included in transactional data. Data that has been partially or completely aggregated calls for a different mode of analysis and should be treated as budgetary data rather than transactional data. This does not mean, however, that several "physical" payments which amount to a single “logical” transaction cannot be represented by a single transaction in transactional data.
+
+Transactional data on OpenSpending includes:
+
+* [DC Vendors and Contractors](http://openspending.org/dc-vendors-contractors)
+
+* [Austrian Development Agency](http://openspending.org/ada/)
+
+Another related type of data deals with the public procurement procedures. Public Procurement data is data about public tenders: what was tendered, for how much, and who won the tender. It can be seen as a subset of transactional data.
+
+Procurement data on OpenSpending includes:
+
+* [Marchés publics au Sénégal](http://openspending.org/marches-publics-senegal/views/liste-des-attributaires)
+
+* [Marchés publics France 2011](http://openspending.org/marches-publics-france-2011)
+
+#### Budgetary data
+
+In budgetary data, expenditures and incomes are aggregated into categories. The goal of this aggregation is to aid the reader in understanding the budget, which is typically a policy document that is used to provide readers with an overview of the government’s most important financial choices. Allocation is typically structured by a classification scheme rather than by the actual recipients of funds.
+
+Budgetary data often jointly presents data on past outcomes and appropriations for a future period. In such a presentation, amounts spent in previous years on a particular sector are used to inform how much should be allocated for the coming budgeting period. Budget information is often based on aggregated data and statistical estimates.
+
+Different regions make different types of budgetary information available, including: Pre-Budget Statements; Executive Budget Proposals; Enacted Budgets; and Citizen's Budgets (simplified versions of the budget for the benefit of citizens).
+
+Budgetary data on OpenSpending includes:
+
+* [Berlin Budget](http://openspending.org/berlin_de)
+
+* [Seville Spending Budget](http://openspending.org/seville-budget)
+
+**Next**: [How does OpenSpending represent data?](../data-model/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/formatting-data/index.md

@@ -0,0 +1,52 @@
+---
+section: help
+lead: true
+title: Formatting data
+authors:
+- Neil Ashton
+---
+OpenSpending expects all data to be in a simple format.
+
+#### CSV
+
+OpenSpending accepts data in a single file format, the Comma-Separated Values (CSV) file. A CSV is a plain text file that represents data as a table, which is similar to a spreadsheet. In a table, each data point is represented by a row, and each data point's properties are represented by a column. CSV files encode tables by giving each row a line in the text file and by separating columns with commas.
+
+CSVs accepted by OpenSpending do not save space by removing redundant values. If your spreadsheet omits any repeated values, those omitted values must be filled in before OpenSpending can use your data. OpenSpending-ready CSVs are also *rectangular*, meaning that they have exactly the same number of columns in each row.
+
+#### The OpenSpending format
+
+CSVs for OpenSpending must have the following properties.
+
+1. One header row. The first row of the CSV file should contain the names of the columns, separated by commas. All other rows are treated as data rows.
+
+2. At least three columns. The bare minimum of columns are an amount, a date (which could be just a year), and a spender or a recipient (which could just be the name of an account).
+
+3. Consistent columns. Each column must consistently represent a single type of value for all rows. (There can be no subheader rows, for example.)
+
+4. Rows are single data points. Rows should contain *only one* transaction or one budget line. Each row must represent a maximum of one time period.
+
+5. No blank rows or cells. Each row should be completely filled in. Some spreadsheets leave redundant data cells blank or have other ways of saving space, but OpenSpending requires each row to be complete on its own.
+
+6. No pre-aggregated totals (e.g. sub-totals or "roll-ups"). OpenSpending will do the maths and compute these automatically.
+
+7. Rows have values that uniquely identify them. Each row must have some column (or combination of column) whose value(s) can be used as an "ID" for the row. Each row's ID must be unique. For example, your data could have a column named "ID" which contains a different number for each row. An easy way to create such an ID column in Excel is to add a new column, write "1" in the top cell of the column, write "2" in the second cell from the top, select both cells, and then click and drag the lower right corner of the selection to the bottom of the spreadsheet.
+
+8. Dates in the correct format. Dates must be in the format "yyyy-mm-dd".
+
+9. Numbers in the correct format. Numbers must contain only digits and an optional period—no commas! (Readable numbers like "12,345.67" should be converted to numbers like “12345.67”.)
+
+The OpenSpending community has gathered some [example spreadsheets](https://drive.google.com/a/okfn.org/#folders/0B_dkMlz2NopEbmRoTExsMDFMR2M) in order to illustrate what "good" and “bad” tabular data looks like. Here are some examples of badly formatted spreadsheets:
+
+* [Many blank cells](https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AvdkMlz2NopEdEtIMFlEVDZXOWdDUEthUTQ0c21aV2c#gid=0) (probably redundant info omitted)
+
+* [Multiple transactions, one row](https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AvdkMlz2NopEdG5kR0kzQ0E5V3BuTS16MndBT3dMdEE#gid=0) (multiple years on one row)
+
+* [Bad numbers](https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AvdkMlz2NopEdEo1Y2p2R0VvdnJvRXMwUVREbHRoLXc#gid=0) (numbers have commas for readability)
+
+Here is a good spreadsheet:
+
+* [Washington, DC](https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AvdkMlz2NopEdDhrZnRkWl9ZX2ZZNVptTzdueWw3emc#gid=0)
+
+**Next**: [Publishing data on the web](../publishing-data)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/gathering-data/index.md

@@ -0,0 +1,16 @@
+---
+section: help
+lead: true
+title: Gathering data
+authors:
+- Neil Ashton
+---
+To add a dataset to OpenSpending, you must first have some data. If you already have it, you can proceed. If not, you need to find it.
+
+Begin your search for data by consulting esources such as the[ School of Data](http://schoolofdata.org/handbook/courses/finding-data/) and the[ Data Journalism Handbook](http://datajournalismhandbook.org/1.0/en/getting_data.html). You can also get ideas on how to go about your search by visiting the[ OpenSpending group](http://datahub.io/group/openspending) on datahub.io, and you can ask questions on the #openspending IRC channel on Freenode.
+
+The data you find will hopefully be in a "machine-readable" format, for example in the form of an Excel spreadsheet or a CSV file. If you find data in a format like PDF or a Word document, it will be very hard to work with, and you might consider simply trying different data!
+
+**Next**: [Formatting data](../formatting-data/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/index.md

@@ -0,0 +1,27 @@
+---
+section: help
+lead: true
+title: The OpenSpending Guide
+authors:
+- Neil Ashton
+---
+The OpenSpending Guide is the manual for OpenSpending, covering the entire process of finding, adding, and using data with OpenSpending in detail.
+
+* Introduction
+    * [What is OpenSpending?](./what-is-openspending)
+    * [What types of financial data can go into OpenSpending?](./financial-data-types)
+    * [How does OpenSpending represent data?](./data-model)
+* Adding Data to OpenSpending
+    * [Overview](./adding-data-overview)
+    * [Gathering data](./gathering-data)
+    * [Formatting data](./formatting-data)
+    * [Publishing data on the web](./publishing-data)
+    * [Creating a dataset on OpenSpending](./creating-dataset)
+    * [Modelling your data in OpenSpending](./modelling-data)
+* Visualizations
+    * [Create a visualization](./create-viz)
+    * [Embed a visualization on your website](./embed-viz)
+
+**Begin**: [What is OpenSpending?](./what-is-openspending/)
+
+**Up**: [OpenSpending guides](../)

examples/openspending/content/help/guide/modelling-data/index.md

@@ -0,0 +1,76 @@
+---
+section: help
+lead: true
+title: Modelling your data in OpenSpending
+authors:
+- Neil Ashton
+---
+To load data into OpenSpending, you must build a *model* of your data. A model specifies how your data translates into terms OpenSpending understands. OpenSpending represents the properties data in terms of *dimensions*. Modelling data consists of listing the dimensions you would like the target OpenSpending dataset to have and specifying how they relate to columns in the source data.
+
+#### Mandatory dimensions: amount and time
+
+Every model needs to have at least two dimensions: an amount and a time. These specify the size of the transaction and the time when the transaction took place. The amount and time are associated with special types of dimensions. An amount is represented by a *measure* dimension, and a time is represented by a *date*. Generic dimensions cannot represent these special values.
+
+When modelling your data, it's not a bad idea to start with the mandatory dimensions. To begin, click the **Dimensions & Measures** tab within your dataset's **Manage the dataset** page.
+
+<a href="http://blog.openspending.org/files/2013/08/image_5-e1375888673131.png"><img src="http://blog.openspending.org/files/2013/08/image_5-e1375888673131.png" alt="image_5" width="600" height="232" class="alignnone size-full wp-image-587" /></a>
+
+Next, click **Add Dimension** to bring up the *Add new dimension* panel. Click the radio button labeled *Date*. You will see the *Name* box automatically fill with "time", as shown below. Click the green **Add** button.
+
+<a href="http://blog.openspending.org/files/2013/08/image_6-e1375888703851.png"><img src="http://blog.openspending.org/files/2013/08/image_6-e1375888703851.png" alt="image_6" width="600" height="428" class="alignnone size-full wp-image-589" /></a>
+
+The next screen you see will provide you some information about the meaning of time. In the drop-down box next to *Column:*, select the column of your data which represents the time value.
+
+<a href="http://blog.openspending.org/files/2013/08/image_7-e1375888730762.png"><img src="http://blog.openspending.org/files/2013/08/image_7-e1375888730762.png" alt="image_7" width="600" height="257" class="alignnone size-full wp-image-590" /></a>
+
+Once you identify the time column, click **Add Dimension** once again to add the amount. This time, select the radio button labeled *Measure*, which will automatically fill in the name "amount", and click **Add**. Choose the column representing the value of the transaction from the drop-down box next to *Column*.
+
+#### The key and compound dimensions
+
+Only one additional dimension is necessary to make the model sufficient: the dimension (or set of dimensions) whose value uniquely identifies each data point, the *key*.
+
+A data point does not need to be identified by the value of a single column. It can be identified by the combination of several in a *compound dimension*. Because keys *can* be compound, the compound dimension type *must* be used to represent them, even if your particular key is not compound.
+
+To add the key dimension, click **Add Dimension** and select the *Dimension* radio button. Enter a name for your key, such as "key", in the name box. Click **Add**. Check the box labeled *Include in unique key* to identify this dimension as part of your key.
+
+Next, take a look at the list of **Fields**, which contains two rows labeled *name *and *label*. A compound dimension can contain an arbitrary number of *fields*, each of which has a name and a type and each of which can be associated with a column in your data. This is the sense in which these dimensions are "compound": they group multiple columns from the source data into a single property of the target dataset.
+
+<a href="http://blog.openspending.org/files/2013/08/image_8-e1375888755790.png"><img src="http://blog.openspending.org/files/2013/08/image_8-e1375888755790.png" alt="image_8" width="600" height="378" class="alignnone size-full wp-image-591" /></a>
+
+A compound dimension requires at least two fields, *name* and *label*, which must respectively be of type *id* and *string*. The dimension's name is used to provide it with a working URL, and the label is used to present it in the user interface.
+
+To create a minimal compound dimension, simply associate the same column of the source data with both *name* and *label*. Choose the appropriate column for each and leave the default types unchanged.
+
+#### Measures and other dimensions
+
+With an amount, time, and key, your model is sufficiently rich. A really complete model, however, will include dimensions for every meaningful property of the source data. Following certain conventions makes this more convenient.
+
+A common pattern in source data is spreading information that identifies entities – groups, accounts, and so on – across multiple columns. Information about an account associated with a transaction may be divided into an "Account" column with an identifying number and an "Account description" column with a verbal description, for example. "Head-account" and "Sub-account" in the image below exhibit this pattern.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_9.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_9.png" alt="image_9" width="533" height="403" class="alignnone size-full wp-image-592" /></a>
+
+OpenSpending's compound dimensions are designed to model this kind of scattered information. To do so, add a new compound dimension and associate each column to one of the dimension's fields. Try to match a human-readable column to *label* and a more terse column to *name*. In the image below, "Head-account" is matched to *name* and "Head-account description" to *label*.
+
+<a href="http://blog.openspending.org/files/2013/08/image_10-e1375888789463.png"><img src="http://blog.openspending.org/files/2013/08/image_10-e1375888789463.png" alt="image_10" width="600" height="371" class="alignnone size-full wp-image-593" /></a>
+
+Some columns of your data are more self-contained, representing particular attributes of each data point. A column which sorts each transaction into some category, for example, is of this type. In the image below, the Reporting Type, Revenue/Expenditure, and Recurrent/Investment columns are like this.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_11.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_11.png" alt="image_11" width="609" height="378" class="alignnone size-full wp-image-594" /></a>
+
+Self-contained columns specifying attributes or categories are best modeled with *attribute* dimensions. An attribute is essentially a dimension with only a single field, which may have any type. To create an attribute, simply select the *Attribute* radio button when adding a dimension.
+
+<a href="http://blog.openspending.org/files/2013/08/image_12-e1375888823415.png"><img src="http://blog.openspending.org/files/2013/08/image_12-e1375888823415.png" alt="image_12" width="600" height="472" class="alignnone size-full wp-image-595" /></a>
+
+#### Wrapping up: saving and loading
+
+When every dimension has been specified and linked to columns in the source data, click **Save Dimensions** to save the model. If anything is wrong with the model, an error message will appear, prompting you to correct its parameters. Otherwise, a message will appear inviting you to return to the dashboard, where you can proceed to load your data.
+
+Once the data has been loaded, the model you have created will be fixed and editing will be disabled. You may therefore wish to test the model before you load. To do this, click **Test a sample** in your data source's row in the dashboard. Wait a few seconds, then reload the page. If you see a message saying COMPLETE with a green background, your model is ready to go. If you see ERRORS, repairs are needed.
+
+<a href="http://blog.openspending.org/files/2013/08/image_13-e1375888848457.png"><img src="http://blog.openspending.org/files/2013/08/image_13-e1375888848457.png" alt="image_13" width="600" height="279" class="alignnone size-full wp-image-596" /></a>
+
+If your model is free of errors, click **Load** to load the source dataset and apply the model. You may then return to the dataset's home page by clicking its name at the top of the screen, where you can proceed to construct visualizations and otherwise play with your data.
+
+**Next**: [Create a Visualization](../create-viz/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/publishing-data/index.md

@@ -0,0 +1,38 @@
+---
+section: help
+lead: true
+title: Publishing data on the web
+authors:
+- Neil Ashton
+---
+Data cannot (yet) be uploaded directly to OpenSpending. In order to be added to the OpenSpending database, data must first be made accessible from the web. This section introduces two convenient ways to publish sets of data online.
+
+#### Google Drive
+
+You can make your data accessible on the web by turning it into a Google Drive spreadsheet.
+
+1. Import your data. Create a new Google Drive spreadsheet, then select *Import...* from the File menu. Select *Replace Spreadsheet*, click **Choose File**, and navigate to your CSV file.
+
+2. Make sure Google Docs doesn't mis-format your data's dates. Select the column that contains dates. Click the *Format* menu and select *Number* -> *More formats* -> *2008-09-26*. Your dates should appear in the prescribed **yyyy-mm-dd** format.
+
+3. Click the *File* menu and select *Publish to the web...*. In the box that appears, click **Start publishing**. Beneath *Get a link to the published data*, select **CSV (comma-separated values)**.
+
+<a href="{{ site.baseurl }}/img/blog/2013/08/image_0.png"><img src="{{ site.baseurl }}/img/blog/2013/08/image_0.png" alt="image_0" width="596" height="578" class="alignnone size-full wp-image-577" /></a>
+
+The URL at the bottom of the box now points to your data.
+
+#### Gist
+
+GitHub Gist is a convenient way to host small quantities of text, including CSV files.
+
+1. Log in to GitHub (or register if you haven't already done so), then navigate to[ gist.github.com](https://gist.github.com/).
+
+2. Click and drag your CSV file from your operating system's file manager onto the GitHub Gist page of your browser. The file's name and contents will appear.
+
+3. Click **Create Public Gist** to be taken to the homepage of your new gist. The raw URL of your data is accessible through the "angle brackets" button in the top right corner of the file.
+
+<a href="http://blog.openspending.org/files/2013/08/image_1-e1375888253802.png"><img src="http://blog.openspending.org/files/2013/08/image_1-e1375888253802.png" alt="image_1" width="600" height="141" class="alignnone size-full wp-image-578" /></a>
+
+**Next**: [Creating a dataset on OpenSpending](../creating-dataset/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/guide/what-is-openspending/index.md

@@ -0,0 +1,14 @@
+---
+section: help
+lead: true
+title: What is OpenSpending?
+authors:
+- Neil Ashton
+---
+OpenSpending is a data sharing community and web application that aims to track every government and corporate financial transaction across the world and to present that data in a useful and engaging form. OpenSpending is an open project maintained by a community of contributors. Anyone interested in spending data of any kind is invited to contribute data to the OpenSpending database, create visualizations using the OpenSpending software, and use the OpenSpending API.
+
+This chapter introduces new OpenSpending contributors to the core concepts of the system. It describes the kind of financial data that OpenSpending supports, and it explains how OpenSpending represents that data.
+
+**Next**: [What types of financial data can go into OpenSpending?](../financial-data-types/)
+
+**Up**: [OpenSpending Guide](../)

examples/openspending/content/help/index.md

@@ -0,0 +1,49 @@
+---
+title: Help
+authors:
+- Anders Pedersen
+redirect_to: "/get-involved/"
+---
+
+Are you looking to get started using OpenSpending? Have a look at some
+of our guides and get rolling! If you have questions,
+[get in touch](../about/contact).
+
+The OpenSpending Guide is available in multiple languages:
+
+* [English](./guide/en)
+* [French](./guide/fr)
+* [Italian](./guide/it)
+* [Nepali](./guide/npl)
+* [Romanian](./guide/rom)
+* [Spanish](./guide/esp)
+
+Many other translations are under development. Check back for new
+translations or
+[help add your language](https://trello.com/c/abfAVgBC/14-permanent-openspending-guide-translation).
+
+#### Video guides
+
+* [OpenSpending Upload, Step 1](http://vimeo.com/43259079#): adding a dataset
+* [OpenSpending Upload, Step 2](http://vimeo.com/43760979): data modelling
+* [How to Fix Errors in OpenSpending](http://vimeo.com/43762097)
+* [Introduction to Spending Stories](http://webtv.journalismfestival.com/doc/1311/raccontare-la-spesa-pubblica.htm)
+* [Talk from "The Big Clean", Prague 2012](http://www.youtube.com/watch?v=ofzU43g5ZpY)
+
+#### Tutorials
+
+* [D3.js Sankey diagrams with the OpenSpending API](http://blog.openspending.org/2013/08/28/d3-sankey/)
+
+#### OpenSpending API
+
+OpenSpending aims to be a simple-to-use platform for developers by
+offering a variety of APIs that can be used to create visualizations,
+reports, and other applications. These help pages describe the APIs
+and provide examples of their use.
+
+* [API conventions](./conventions)
+* [Aggregate API](./aggregate)
+* [REST resources](./rest)
+* [Full-text search API](./search)
+* [Personal tax API](./tax)
+* [Dataset loading API](./loading)

examples/openspending/content/help/links.md

@@ -0,0 +1,25 @@
+---
+section: help
+lead: true
+title: Key Community Resources
+authors:
+- Neil Ashton
+---
+
+OpenSpending Community key shared resources.
+
+#### Documents
+
+* [Community Google Drive folder](https://drive.google.com/a/okfn.org/#folders/0B6R8dXc6Ji4JUWs5UDdzSzU2UG8)
+
+#### Data Stuff
+
+* [Tracker for data to load](https://github.com/openspending/datatoload)
+
+#### Code
+
+[OpenSpending on GitHub](https://github.com/openspending)
+
+* [Core application](https://github.com/openspending/openspending)
+* [JavaScript library](https://github.com/openspending/openspendingjs)
+* [Satellite site template](https://github.com/openspending/satellite-template)

examples/openspending/content/help/loading.md

@@ -0,0 +1,18 @@
+---
+section: help
+lead: true
+title: Dataset loading API
+authors:
+- Neil Ashton
+---
+
+#### OpenSpending Loading API
+
+Users can load datasets (or add sources to them) by making a *POST* request to ``https://openspending.org/api/2/new`` (notice *https*) with the following url parameters:
+
+* *csv_file* - A **url** to the csv file to me imported for the dataset
+* *metadata* - A **url** to the json file with dataset metadata (name, currency, etc.) and the model. Views can also be defined in this file. Take a look at [a sample json file](https://dl.dropbox.com/u/3250791/sample-openspending-model.json) to see how it should be structured (the value for *mapping* is the model - how the csv file should be cast into dataset dimensions, and the value for *dataset* is the metadata itself). To gain a better understanding of how to do the mapping, take a look at [the corresponding csv file](http://mk.ucant.org/info/data/sample-openspending-dataset.csv).
+
+Along with these two an api key must be provided in the header of the request. For more details see [API Conventions](/help/conventions/).
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/permissions.md

@@ -0,0 +1,23 @@
+---
+section: help
+lead: true
+title: Permissions API
+authors:
+- Tryggvi Björgvinsson
+---
+OpenSpending allows users to check for their permissions on a given dataset via an API call. The response will provide the authenticated user's permission on as true or false values for *CRUD* (create, read, update, and delete). This API call mainly exists to allow software that uses the API (e.g. [the loading API](/help/api/loading)) to save bandwidth with big dataset updates.
+
+For example if you as a developer are building a loading script that users of OpenSpending can use to download data from a location and update datasets in OpenSpending you might first run a check for permissions based on their [API key](http://community.openspending.org/help/api/conventions) before starting to download the updates (so you can skip it if they're not authorized.
+
+The permission API works as follows. Make a *GET* request (wih user authenticated with the API key) to:
+
+    /api/2/permissions?dataset=[dataset_name]
+
+The response will be single json object with four properties, *create*, *read*, *update*, and *delete*. The value of each property is a boolean (true or false) that indicates if the authenticated user has that permission for the provided dataset:
+
+    {
+        "create": false,
+        "read": true,
+        "update": false,
+        "delete": false
+    }

examples/openspending/content/help/rest.md

@@ -0,0 +1,197 @@
+---
+section: help
+lead: true
+title: REST resources
+authors:
+- Neil Ashton
+---
+OpenSpending pages generally support multiple representations, at least
+a user-facing HTML version and a JSON object that represents the contained
+data. For various technical and non-technical reasons, most of the data is
+read-only.
+
+Content negotiation can be performed either via HTTP ``Accept`` headers or
+via suffixes in the resource URL. The following types are generally
+recognized:
+
+* **HTML** (Hyptertext Markup), MIME type ``text/html`` or any value not
+  otherwise in use, suffix ``.html``. This is the default representation.
+* **JSON** (JavaScript Object Notation), MIME type ``application/json`` and
+  suffix ``.json``.
+* **CSV** (Comma-Separated Values), MIME type ``text/csv`` and suffix
+  ``.csv``. CSV is only supported where listings can be exported with some
+  application-level meaning.
+
+The key resources in OpenSpending are datasets, entries, dimensions, and
+dimension members. Each of these has a listing and an entity view that can
+be accessed.
+
+#### Listing datasets
+
+    GET /datasets.json
+
+All datasets are listed, including their core metadata. Additionally, certain
+parameters are given as facets (i.e. territories and languages of the
+datasets). Both ``territories`` and ``languages`` can also be passed in as
+query parameters to filter the result set. Supported formats are HTML, CSV and JSON.
+
+    "territories": [
+      /* ... */
+      {
+        "count": 2,
+        "url": "/datasets?territories=BH",
+        "code": "BH",
+        "label": "Bahrain"
+      },
+      /* ... */
+    ],
+    "languages": /* Like territories. */
+    "datasets": [
+      {
+        "name": "cra",
+        "label": "Country Regional Analysis v2009",
+        "description": "The Country Regional Analysis published by HM Treasury.",
+        "currency": "GBP"
+      },
+      /* ... */
+    ]
+
+#### Getting dataset metadata
+
+    GET /{dataset}.json
+
+Core dataset metadata is returned. This call does not have any
+parameters. Supported formats are HTML and JSON.
+
+    {
+      "name": "cra",
+      "label": "Country Regional Analysis v2009",
+      "description": "The Country Regional Analysis published by HM Treasury.",
+      "currency": "GBP"
+    }
+
+Another call is available to get the full model description of
+the dataset in question, which includes the core metadata and also
+a full description of all dimensions, measures, and views. The
+format for this is always JSON::
+
+    GET /{dataset}/model.json
+
+#### Listing dataset dimensions
+
+    GET /{dataset}/dimensions.json
+
+A listing of dimensions, including type, description, and attribute
+definitions is returned. This call does not have any parameters.
+Supported formats are HTML and JSON.
+
+    [
+      {
+        "name": "from",
+        "html_url": "http://openspending.org/ukgov-finances-cra/from",
+        "label": "Paid from",
+        "key": "from",
+        "attributes": {
+          "gov_department": {
+            "column": null,
+            "facet": false,
+            "constant": "true",
+            "datatype": "constant",
+            "end_column": null
+          },
+          "name": {
+            "column": "dept_code",
+            "facet": false,
+            "constant": null,
+            "datatype": "string",
+            "end_column": null
+          },
+          "label": {
+            "column": "dept_name",
+            "facet": false,
+            "constant": null,
+            "datatype": "string",
+            "end_column": null
+          }
+        },
+        "type": "compound",
+        "description": "The entity that the money was paid from"
+      },
+      /* ... */
+    ]
+
+#### Listing dimension members
+
+    GET /{dataset}/{dimension}.json
+
+The returned JSON representation contains the dimension metadata,
+including type, label, description and attribute definitions.
+
+    {
+      "name": "from",
+      "html_url": "http://openspending.org/ukgov-finances-cra/from",
+      "label": "Paid from",
+      "key": "from",
+      "attributes": {
+        "gov_department": {
+          "column": null,
+          "facet": false,
+          "constant": "true",
+          "datatype": "constant",
+          "end_column": null
+        },
+        "name": {
+          "column": "dept_code",
+          "facet": false,
+          "constant": null,
+          "datatype": "string",
+          "end_column": null
+        },
+        "label": {
+          "column": "dept_name",
+          "facet": false,
+          "constant": null,
+          "datatype": "string",
+          "end_column": null
+        }
+      },
+      "type": "compound",
+      "description": "The entity that the money was paid from"
+    }
+
+This call's return includes dimension metadata, but it may be too expensive
+to call for just this aspect.
+
+#### Getting dimension members
+
+    GET /{dataset}/{dimension}/{name}.json
+
+This will return the data stored on a given member ``name`` of the
+``dimension``, including its ``name``, ``label``, and any other
+defined attributes.
+
+    {
+      "id": 2,
+      "name": "10",
+      "label": "Social protection",
+      "description": "Government outlays on social protection ...",
+      "level": "1"
+    }
+
+#### Listing entries in a dataset
+
+Listing all the entries in a dataset (and offering export functionality)
+is handled by the full-text search. See [the search API](../search).
+
+### Getting an entry
+
+    GET /{dataset}/entries/{id}.json
+
+This will return a full representation of this entry, including all
+measures and all attributes of all dimensions. The entry ``id`` is a
+semi-natural key derived from dataset metadata which should be stable
+across several loads.
+
+A CSV representation is available but will only have one row.
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/search.md

@@ -0,0 +1,93 @@
+---
+section: help
+lead: true
+title: Full-text search API
+authors:
+- Neil Ashton
+---
+OpenSpending supports full-text search as a research tool for
+everyone who wants to investigate the spending information kept
+in our database.
+
+It is important to note, however, that search is always performed
+on individual entries. More abstract concepts (e.g. "all
+health spending in a country over a given year") would mostly be the
+result of adding up many individual entries. If your use case
+requires that you access such concepts, you may want to look at
+the [aggregation API](../aggregate) instead.
+
+#### Basic call and parameters
+
+    GET /api/2/search?q=<query>
+
+Calls will return a set of fully JSON serialized entries, query
+statistics, and, depending on the other parameters, other data such as
+facets.
+
+The following parameters are recognized:
+
+* ``q``
+  Query string. Will usually search a composite text field but can
+  be limited to a specific field (i.e. a dimension, attribute, or measure)
+  with ``field:value``. Boolean operators such as OR, AND, and ±term can also be used.
+
+* ``dataset``
+  Specifies a dataset name to search in. While searching across multiple
+  datasets is supported, this parameter can be used to limit the scope and
+  increase performance. It can be used multiple times or multiple
+  dataset names can be separated with pipe symbols.
+
+* ``category``
+  The dataset category can be used to filter datasets by their type,
+  e.g. limiting the output to only transactional expenditure (and
+  excluding any budget items). Valid values include ``budget``,
+  ``spending``, and ``other``.
+
+* ``stats``
+  Includes solr statistics on measures, namely the average, mean, and
+  standard deviations. This is generated through the indexed data and
+   can differ marginally from the
+  results of the aggregator due to floating point inaccuracies.
+  Note that aggregations
+  across datasets with different currencies (or even the same currency
+  across different years) are possible but must be avoided.
+
+* ``filter``
+  Apply a simple filter of the format ``field:value``. Multiple filters
+  can be joined through pipes, e.g. ``fieldA:value|fieldB:value``.
+
+* ``page``
+  Page number for paginated results. Defaults to ``1``.
+
+* ``pagesize``
+  Size of a page for paginated results. Defaults to ``10000``.
+
+* ``facet_field``
+  A field to facet the search by, i.e. give all the distinct values of
+  the field in the result set with the count of how often each occurred.
+
+* ``facet_page``, ``facet_pagesize``
+  Works analogously to the ``page`` and ``pagesize`` parameters but applies
+  to facets instead.
+
+* ``expand_facet_dimensions``
+  When a compound dimension name is used for a facet, this will return a
+  full representation of this dimension value for each value.
+
+If an error is detected, the system will return a simple JSON response
+with a list of ``errors`` describing the fault.
+
+### Solr query syntax
+
+OpenSpending uses Apache Solr for full-text indexing. Some search
+parameters are passed directly to Solr:
+
+    GET /api/2/search?q=money%20measure:[min%20TO%20max]&fq=dimension:value
+
+Some useful resources to explore the query language of Solr include:
+
+* [Solr Common Query Parameters](http://wiki.apache.org/solr/CommonQueryParameters)
+* [Lucene Query Parser Syntax](http://lucene.apache.org/java/3_4_0/queryparsersyntax.html)
+* [Solr Query Syntax](http://wiki.apache.org/solr/SolrQuerySyntax) (Advanced)
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/tax.md

@@ -0,0 +1,50 @@
+---
+section: help
+lead: true
+title: Personal tax API
+authors:
+- Neil Ashton
+---
+The tax share API estimates a household's tax contribution based on simple
+proxy data. The estimate allows for both direct tax (including income tax,
+national insurance and council tax) and indirect tax (including VAT, alcohol
+and tobacco duty, and fuel duty).
+
+    GET http://openspending.org/api/mytax?income=N
+
+The basic call accepts a variety of parameters, most of which are optional:
+
+* ``income`` (required)
+  Total household income, including all pension and benefits. This is
+  used to estimate total tax paid, including both direct and indirect
+  taxation.
+
+* ``spending``
+  Total spending on consumption.
+
+* ``smoker``
+  yes/no
+
+* ``drinker``
+  yes/no
+
+* ``driver``
+  yes/no
+
+This will generate a simple JSON response of the following form:
+
+    {
+      "alcohol_tax": 153.04239230064161,
+      "explanation": [
+        "This household income falls between national average income decile 1 (which has average gross household income of 9219.00, and pays 1172.00 in direct tax, 1016.00 in VAT, 1101.00 in smoking taxes, 288.00 in alcohol-related taxes, 150.00 in car-related taxes, and 349.00 in other indirect taxes), and decile 2 (which has average gross household income of 13583.00, and pays 1368.00 in direct tax, 969.00 in VAT, 1085.00 in smoking taxes, 310.00 in alcohol-related taxes, 167.00 in car-related taxes, and 289.00 in other indirect taxes).",
+        "Therefore, a household with an income of 10000.00 pays approximately 1207.08 in direct tax and 2888.97 in total indirect tax."
+      ],
+      "tax": 4096.0439963336394,
+      "tobacco_tax": 291.93721356553618,
+      "car_related_tax": 338.26214482126488,
+      "total_direct_tax": 1207.076993583868,
+      "vat": 1098.1365719523374,
+      "total_indirect_tax": 2888.9670027497709
+    }
+
+**Up**: [OpenSpending API](../)

examples/openspending/content/help/taxman.mdown

@@ -0,0 +1,35 @@
+# The Where Does My Money Go - Daily Bread, how we calculate the numbers
+
+We are often asked, how we calculate the numbers in Daily Bread.
+
+Information on the sources of data which we use for the Daily Bread UK tax calculator can be found on the Where Does my Money Go Site:
+
+<http://wheredoesmymoneygo.org/sources.html>
+
+It is possible to build your own version of this for your country - here's what you would need to do so...
+
+<img alt="" src="http://farm9.staticflickr.com/8035/7899326662_d471556831_z.jpg" title="Where Does My Moeny Go - Daily Bread Tax Calculator" class="alignnone" width="640" height="480" />
+
+# Build your own tax calculator - the recipe
+
+## Raw ingredients
+
+* **Information on the budget with functional classifications** (in order to be digestible to the average reader, we suggest that you don't have more than 10 top level items or there will probably be too much information for the user to take in).
+* Information on the tax rates and how they are calculated
+* Information on any overriding concerns e.g. if you are a smoker or a driver, is your tax rate drastically affected and are you interested in reflecting this in your tax calculator?
+
+### If you want your tax calculator to resemble Where Does My Money Go? You will also need...
+
+* Images to reflect the functional areas of the budget in your country N.B. If your budget uses the UN's [COFOG](http://en.wikipedia.org/wiki/Classification_of_the_Functions_of_Government) classifications then it will align with the UK's budget and you will be able to use the images from Where Does My Money Go? These images live in the [Github repo](https://github.com/openspending/wheredoesmymoneygo.org/tree/master/icons).
+
+## Instructions
+
+*You'll need a developer with some knowledge of JavaScript (node.js) and someone with a good understanding of the tax system of your country*.
+
+Our tax calculator runs off an API which we affectionately call ['taxman'](https://github.com/openspending/taxman), which can be adapted to accommodate different tax rates, output types (e.g. individual daily, monthly or yearly tax contributions) and other factors such as indirect taxes, which you may also want to reflect in your calculator.
+
+## Useful links
+
+* [Where Does My Money Go? - The Daily Bread](http://wheredoesmymoneygo.org/dailybread.html)
+* [Taxman API](https://github.com/openspending/taxman)
+* The Taxman API lives online on [OpenSpending](The TaxMan API lives at: http://taxman.openspending.org/)