[examples/openspending] - openspending v0.2 (#907)

* [examples/openspending] - openspending v0.2

* [examples/openspending][m] - fix build

* [examples/openspending][xs] - fix build

* [examples/openspending][xs] - add prebuild step

* [examples/openspending][m] - fix requested by demenech

* [examples/openspending][sm] - remove links + fix bug

examples/openspending/content/resources/eu/acronyms.md

@@ -0,0 +1,40 @@
+---
+title: List of Acronyms
+---
+
+| **ACP** | African, Caribbean and Pacific (countries) |
+| **CAP** | Common Agricultural Policy |
+| **CF** | Cohesion Fund |
+| **CFP** | Common Fisheries Policy |
+| **CSO** | Civil Society Organisation |
+| **DG** | Directorate-General |
+| **EAFRD** | European Agricultural Fund for Rural Development |
+| **EAGF** | European Agricultural Guarantee Fund |
+| **EDF** | European Development Fund |
+| **ECA** | European Court of Auditors |
+| **EFF** | European Fisheries Fund |
+| **EMFF** | European Maritime and Fisheries Fund |
+| **ERCAS** | European Research Centre for Anti-Corruption and State-Building |
+| **ERDF** | European Regional Development Fund |
+| **ESIF** | European Structural and Investment Funds |
+| **EU** | European Union |
+| **FIFG** | Financial Instrument for Fisheries Guidance |
+| **FTS** | Financial Transparency System |
+| **GNI** | Gross National Income |
+| **ICIJ** | International Consortium of Investigative Journalists |
+| **IMF** | International Monetary Fund |
+| **IMP** | Integrated Maritime Policy |
+| **MFF** | Multiannual Financial Framework |
+| **OCTs** | Overseas Countries and Territories |
+| **OLAF** | European Anti-Fraud Office |
+| **TBIJ** | The Bureau of Investigative Journalism |
+| **TED** | Tenders Electronic Daily |
+| **TFUE** | Treaty on the Functioning of the European Union |
+| **TI-EU** | Transparency International EU Office |
+| **UN** | United Nations |
+| **VAT** | Value-added Tax |
+
+- [Return to Beginning](../)
+- [Next: Glossary](../glossary/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/budget.md

@@ -0,0 +1,77 @@
+---
+title: How is the EU Budget Set Up?
+---
+
+Unlike most national budgets which are only set up on an annual basis, the EU budget is set up through two main instruments: the Multiannual Financial Framework (MFF) for the long term, and a more standard annual budget regulation.
+
+The Multiannual Financial Framework
+-----------------------------------
+
+The MFF is an instrument which sets the maximum annual amounts - or “ceilings” - which the EU may spend in different political fields - or “headings” - for a period of at least 5 years. In practice, the MFF is adopted for a period of 7 years and the current one is applicable for the years 2014-2020.[^ftnt2]
+
+The MFF takes the form of a Council regulation, adopted following a special legislative procedure called the consent procedure:
+
+1.  The European Commission submits a proposal for an MFF regulation;
+2.  The European Parliament accepts or rejects the proposal by an absolute majority vote - but it cannot amend the proposal;
+3.  After the approval of the European Parliament, the Council adopts the MFF Regulation by a unanimous vote.
+
+Through the MFF regulation, ceilings for each year concerning commitment appropriations and payment appropriations are determined for each of the MFF headings. Headings change from one MFF to the other, making comparisons difficult over the years.
+
+For the period 2014/2020, the headings are:
+
+-   Smart and inclusive growth including:
+-   Competitiveness for growth and jobs;
+-   Economic, social and territorial cohesion;
+-   Sustainable growth: natural resources;
+-   Security and citizenship;
+-   Global Europe;
+-   Administration;
+-   Compensation.
+
+The framework also sets up ceilings for a number of instruments established outside the general headings and known as the Flexibility and Special Instruments, such as (1) the Emergency Aid Reserve; (2) the European Union Solidarity Fund; (3) the Flexibility Instrument;[^ftnt3] (4) the European Globalisation Adjustment Fund; (5) the Contingency Margin; (6) the Global Margin for commitments for growth and employment, in particular youth employment. These instruments enable the EU to mobilise the necessary funds to react to unforeseen events (financial crisis, emergency situations).
+
+One specific instrument, the European Development Fund (EDF) is managed outside the framework of the MFF and governed by its own set of rules. The EDF provides development aid to African, Caribbean and Pacific (ACP) countries, as well as to Overseas Countries and Territories (OCTs). It is based on an ACP-EU Partnership Agreement known as the “Cotonou Agreement”, which is revised every five years, and it is financed by direct contributions from EU Member States. The resources for the current EDF amount to €30.5 billion for the period 2014 - 2020.
+
+### What are Commitment and Payment Appropriations?
+
+In the EU annual budget and in the MFF regulations, authorised expenditure amounts are expressed in both **commitment** and **payment appropriations**.
+
+-   **Commitments** are legal pledges to provide finance, provided that certain conditions are fulfilled. The EU commits itself to reimburse its share of the costs of an EU-funded project when the project is completed.
+-   **Payments** are cash or bank transfers to beneficiaries. Today’s commitments are tomorrow’s payments.[^ftnt4]
+
+This means for instance that 2015 payments are actually 2014 commitments. In practice, this implies that the EU does not finance a project in advance. Someone, a Member State in most of the cases, would have to provide the funds beforehand before being reimbursed by the EU.
+
+The ceilings of the 2014-2020 MFF are **€959 988 million** in commitment appropriations and **€908 400 million** in payment appropriations. This represents a decrease compared to the 2007-2013 MFF, whose ceiling in commitment appropriations was up to €994 176 million. The main areas concerned by this decrease are the Cohesion Policy, under the heading Smart and Inclusive Growth (-8%) and the Agriculture and Rural Development Policy, under the heading Sustainable Growth: Natural Resources (-11%).[^ftnt5]
+
+The EU Annual Budget
+--------------------
+
+Every year, a regulation is adopted to set up the annual budget of the EU. This budget is based on the MFF and the procedure followed is the ordinary legislative procedure.[^ftnt6] One of the guiding principles for establishing the annual budget is that *“the revenue and expenditure shown in the budget must be in balance”*.[^ftnt7]
+
+**Figure 1: EU annual budget regulation adoption procedure**![Annual Budget.png](../img/image08.png)
+
+Source: European Commission | [SVG version available here](https://drive.google.com/open?id=0BxUOTiQczyxHaDBqSmdDR0FTRlE&authuser=0)
+
+The annual budget regulation is divided into several sections detailing, for each of the EU institutions, the relevant revenue, expenditure and staff allocated. The largest section of the regulation concerns the European Commission, which is responsible for managing the EU policies.
+
+For the year 2015, the budget amounts to:
+
+-   €145 321 million in commitment appropriations;
+-   €141 214 million in payment appropriations (this represents an increase of 1,57% compared to the 2014 EU annual budget);
+-   €141 214 million in budgetary revenue.[^ftnt8]
+
+It is possible to amend the budget after its adoption.
+
+[Annual activity reports](http://ec.europa.eu/atwork/synthesis/aar/index_en.htm) are prepared every year by each Directorate-General (DG) and contain the DG’s annual accounts and financial reports. These reports are summarised in a synthesis report, sent to the European Parliament, the Council and the European Court of Auditors (ECA).
+
+In addition, an annual report on budgetary and financial management is published every year by the Commission.[^ftnt9] 
+
+In November every year, the Court of Auditors publishes public reports on the implementation of the EU budget and the EDF by the Commission. Based on these reports, the European Parliament expresses an opinion on the implementation of the budget by the European Commission. If this opinion is positive, it means that the Parliament has granted discharge to the Commission for this financial year.[^ftnt10]
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: The EU Budget at a Glance](../overview/)
+- [Next: Where Does the Money Come From?](../revenue/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/datasets.md

@@ -0,0 +1,19 @@
+---
+title: Annex - The Open Data Audit of EU Funds
+---
+
+As we have seen, detailed information on EU spending and beneficiaries managed directly by the Commission can be found on the EU FTS, but it only concerns 20% of the EU Budget. For the remaining 80%, there is no central place to go and one has to look at every external entity spending EU money.
+
+In order to help in that task, we have set up the **Open Data audit of EU funds**. The aim of this census is to identify all existing datasets containing information on EU funds beneficiaries, and to assess whether or not data are available as open data.
+
+* * * * *
+
+{% for dataset in site.eu_datasets %}
+[{{dataset.title}}]({{ dataset.url }})
+{% endfor %}
+
+- [Return to Beginning](../)
+- [Prev: Recommendations and Next Steps](../recommendations/)
+- [Next: Annex - Legal Basis for the Establishment of the EU budget](../legal-basis/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/diagrams.md

@@ -0,0 +1,14 @@
+---
+title: Diagrams in editable version
+---
+
+![](../img/image07.png)
+
+* * * * *
+
+![](../img/image00.png)
+
+* * * * *
+
+{% include_relative nav.md %}
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/executive-summary.md

@@ -0,0 +1,9 @@
+---
+title: Executive Summary
+---
+
+
+
+* * * * *
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/expenditure.md

@@ -0,0 +1,41 @@
+---
+title: Who Manages and Spends the Money, and How?
+---
+
+The European Commission has the overall responsibility of the implementation of the EU budget, but does not directly manage all EU funds. Indeed, these funds can be spent in several different ways:
+
+-   Through **centralised management**: the money is managed directly by the European Commission (e.g. through the selection of contractors or the award of grants) - this is **centralised direct management** - or indirectly by EU executive agencies, decentralised agencies or national agencies - this is **centralised indirect management**;
+-   Through **shared management** (this method accounts for around 80% of the EU budget): there is a delegation of budget implementation to EU Member States;
+-   Through **decentralised management**: there is a delegation of budget implementation to non-EU countries;
+-   Through **joint management**: there is a delegation of budget implementation to intergovernmental and international organisations.
+
+One fund can be spent in different ways (for example, through centralised and shared management). **For each fund, a specific regulation is adopted which defines the rules related to its implementation.**
+
+****
+
+**Figure 3****. Budget Management Overview**
+
+****
+
+![](../img/image05.png)
+
+Source: [European Commission](http://ec.europa.eu/budget/explained/management/managt_who/who_en.cfm)
+
+Ultimately, a large number of entities are involved in the spending of the EU budget.
+
+-   **The Commission itself and its agencies.**In that case, money will be directly spent through grants, contracts, or administrative expenditures.
+-   **National and regional authorities in EU Member States;**
+-   **Authorities in non-EU countries** which receive financial support from the EU;
+-   **International organisations** such as the United Nations or the World Bank.
+
+As a general rule, the co-financing principle usually applies to EU spending, especially when it comes to shared management. Through this rule, part of the cost of an action or the running costs of an entity is borne by the beneficiary of the grant or by contributors other than the European Commission, with the aim to make beneficiaries responsible for the operational and financial viability of their projects. Some flexibility nevertheless exists to allow for the full financing of some projects with EU funds.
+
+Other rules usually applicable to funding projects with EU money are: the non-profit rule; the non-retroactivity rule and the non-cumulative rule.[^ftnt15]
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: Where Does the Money Come From?](../revenue/)
+- [Next: Where Does the Money Go?](../funds/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/footnotes.md

@@ -0,0 +1,66 @@
+
+[^ftnt1]: For example, see this analysis from the Robert Schuman Foundation on recent opinion polls: [http://www.robert-schuman.eu/en/european-issues/0333-overcoming-democratic-breakdown-in-the-european-union](http://www.robert-schuman.eu/en/european-issues/0333-overcoming-democratic-breakdown-in-the-european-union) 
+
+[^ftnt2]: See [Council regulation n°1311/2013 of 2 December 2013 laying down the multiannual financial framework for the years 2014-2020](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2013.347.01.0884.01.ENG).
+
+[^ftnt3]: The Flexibility Instrument was used to finance the Juncker Plan: [http://ec.europa.eu/priorities/jobs-growth-investment/plan/docs/an-investment-plan-for-europe\_com\_2014\_903\_en.pdf.](http://ec.europa.eu/priorities/jobs-growth-investment/plan/docs/an-investment-plan-for-europe_com_2014_903_en.pdf.)
+
+[^ftnt4]: See [the EU Funding Glossary](http://ec.europa.eu/budget/funding/information/eu-funding-glossary_en).
+
+[^ftnt5]: See [the Assembly of European Regions briefing note](http://www.aer.eu/fileadmin/user_upload/MainIssues/Future_of_Europe/EU_Budget_Review/MFF-briefing-note.pdf).
+
+[^ftnt6]: [The ordinary legislative procedure step by step.](http://www.europarl.europa.eu/aboutparliament/en/20150201PVL00004/Legislative-powers)
+
+[^ftnt7]: Article 310 of the Treaty on the Functioning of the European Union (TFEU).
+
+[^ftnt8]: [Definitive adoption (EU, EURATOM) 2015/339 of the European Union's general budget for the financial year 2015](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L:2015:069:FULL)
+
+[^ftnt9]: The 2014 report is available online:
+
+[^ftnt10]: [Reports of the ECA.](http://www.eca.europa.eu/en/Pages/AuditReportsOpinions.aspx)
+
+[^ftnt11]: [Council Decision of 7 June 2007 on the system of the European Communities' own resources.](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32007D0436)
+
+[^ftnt12]: [2014 Council Decision on the system of own resources of the European Union.](http://register.consilium.europa.eu/doc/srv?l=EN&f=ST%205602%202014%20INIT)
+
+[^ftnt13]: As the UK rebate, lump sum payments and reduced VAT call rates are correction mechanisms, that is to say measures taken to compensate Member States whose contribution to the EU budget is perceived as being too high compared to their relative wealth and the benefits they get out of the EU budget.
+
+[^ftnt14]: [Council regulation (EU) 2015/323 of 2 March 2015 on the financial regulation applicable to the 11th European Development Fund](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2015.058.01.0017.01.ENG)
+
+[^ftnt15]: See for example [this presentation](http://ec.europa.eu/chafea/documents/health/calls/2013/Workshop_on_Joint_Action_2013/Financial_Managment_JA%20Workshop_December_2012.pdf).
+
+[^ftnt16]: [The 12 highlights for the 2014/2020 MFF.](http://ec.europa.eu/budget/mff/highlights/index_en.cfm#funding)
+
+[^ftnt17]: Europe 2020 is the 10-year growth strategy of the European Union for the period 2010-2020. See [http://ec.europa.eu/europe2020/index\_en.htm](http://ec.europa.eu/europe2020/index_en.htm).
+
+[^ftnt18]: [European Commission ESIF webpage.](http://ec.europa.eu/contracts_grants/funds_en.htm)
+
+[^ftnt19]: Initial amount was €74 928 million but in accordance with regulation 1303/2013 (art. 92(6) and 92(7)), parts of this amount have been transferred to the Connecting Europe Facility and to the Fund for European Aid to the Most Deprived.
+
+[^ftnt20]: The EAGF is the fund financing direct payments to farmers and measures regulating or supporting the agricultural market. As the EAFRD, it is managed by DG AGRI, but represents a more significant share of the EU budget (around 80% of the CAP budget).
+
+[^ftnt21]: Source: Brigitte Alfter, cofounder of FarmSubsidy.org and Wobbing.eu.
+
+[^ftnt22]: For example, concerning the exact amount of the five ESIF for the 2014-2020 period, one EU publication mentions a number of [366.9 billion euros](http://ec.europa.eu/contracts_grants/pdf/synergies_beneficiaries_en.pdf) whereas another one refers to a total of [351.8 billion euros](http://ec.europa.eu/regional_policy/sources/docgener/panorama/pdf/mag48/mag48_en.pdf).
+
+[^ftnt23]: It should be noted, however, that DG BUDGET, the EU Directorate-General for Budget, was responsive to requests for information sent by the authors of this report.
+
+[^ftnt24]: The Directive  2012/17/EU introduces a unique identifier for European companies. However, there is no provision for that unique identifier to be used in data on EU funds beneficiaries. See [http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32012L0017](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32012L0017)
+
+[^ftnt25]: [2012 financial regulation, Articles 34 and 35](http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:298:0001:0096:EN:PDF).
+
+[^ftnt26]: The threshold can be set by Member States, see article 12 of regulation 1306/2013: [http://ec.europa.eu/agriculture/cap-funding/beneficiaries/shared/index\_en.htm](http://ec.europa.eu/agriculture/cap-funding/beneficiaries/shared/index_en.htm). The 2014 thresholds are available [here.](http://www.google.com/url?q=http%3A%2F%2Fec.europa.eu%2Fagriculture%2Fcap-funding%2Fbeneficiaries%2Fshared%2Fthreshold_en.pdf&sa=D&sntz=1&usg=AFQjCNH5M8AxntJmIsoWgMVA4HrWN4jnAA)
+
+[^ftnt27]: For a good introduction to data journalism, see Gray, J., Bounegru, L. and Chambers, L. (eds.) The Data Journalism Handbook (O'Reilly Media, 2012), co-edited by Open Knowledge and the European Journalism Center and freely available [online.](http://datajournalismhandbook.org/)
+
+[^ftnt28]: For an in depth-review of journalism investigation on EU funds up to 2012, see: European Commission, Deterrence of fraud with EU funds through investigative journalism in EU-27, 2012,
+
+[^ftnt29]: ICIJ, €6 billion in subsidies fuel Spain’s ravenous fleet, October 2011, [online.](http://www.icij.org/project/looting-seas-ii/nearly-eu6-billion-subsidies-fuel-spains-ravenous-fleet)
+
+[^ftnt30]: ICIJ,  [methodology](http://www.icij.org/project/looting-seas-ii/subsidy-methodology).
+
+[^ftnt31]: European Commission, Deterrence of fraud with EU funds through investigative journalism in EU-27, p. 227,2012:[http://www.journalismfund.eu/sites/default/files/EP%20Study%20-%20Deterrence%20of%20fraud.pdf](http://www.journalismfund.eu/sites/default/files/EP%20Study%20-%20Deterrence%20of%20fraud.pdf).
+
+[^ftnt32]: See, James T Hamilton, Democracy's Detectives: The Economics and Impacts of Investigative Reporting (forthcoming).
+
+[^ftnt33]: [TFUE](http://eur-lex.europa.eu/legal-content/FR/TXT/?uri=CELEX:12012E/TXT).

examples/openspending/content/resources/eu/funds.md

@@ -0,0 +1,334 @@
+---
+title: Where Does the Money Go?
+---
+
+The European Union has a budget to pay for policies carried out at the European level and for the institutions deciding, managing and implementing these policies (the Commission and its Directorates-General, the Parliament, the agencies, the Council, etc.).
+
+The spending priorities are set in the MFF when defining the amount of the funds under its headings. For the 2014-2020 MFF, the priorities are creating growth and jobs and reducing economic gaps between the EU's various regions.[^ftnt16] Agriculture, rural development, fisheries and environmental protection account for a major share of the budget.
+
+Each heading is then split into programmes and funds which finance the implementation of EU policies. The final beneficiaries can be any person or organisation (company, research organisation, local council, etc.) from the EU Member States and beyond.
+
+As the programmes and funds are managed by various entities belonging or not to the EU institutions, **there is currently no central database of all the EU spending beneficiaries.** 
+
+**The European Structural and Investment Funds (ESIF)**
+
+A very large amount of the EU budget is spent through 5 funds known as the **European Structural and Investment Funds (ESIF).**Indeed, with a total of 366.9 billion euros for the 2014-2020 period, the ESIF make up more than one third of the EU’s budget. To these funds, in addition to the specific regulations regarding their implementation, applies a single set of common rules defined in the [Common Provisions regulation](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv:OJ.L_.2013.347.01.0320.01.ENG), with the objectives of:
+
+-   Establishing a clear link with the Europe 2020 Strategy;[^ftnt17]
+-   Improving coordination;
+-   Ensuring consistent implementation;
+-   Making access to the funds as straightforward as possible for those who may benefit from them.[^ftnt18]
+
+The ESIF are managed under the shared management mechanism, which provides that Members States are co-responsible for the funds and, as such, must comply with specific control and audit procedures.
+
+In 2014, Member States had to prepare and negotiate with the Commission so-called Partnership Agreements (strategic plans with their investment priorities covering the five ESIF). Then, they adopted Operational Programmes, which break down the investment priorities and objectives of the Partnership Agreements into concrete actions. The Operational Programmes are managed by the managing/national authorities appointed by each Member State. Applications for funding are then submitted to the national or regional authority managing the relevant programme.
+
+The  5 ESIF  for the 2014-2020 period are:
+
+-   The Cohesion Fund (CF);
+-   The European Agricultural Fund for Rural Development (EARDF);
+-   The European Maritime and Fisheries Fund (EMFF)
+-   The European Regional Development Fund (ERDF);
+-   The European Social Fund (ESF).
+
+Below are details for each of the 5 ESIF.
+
+The Cohesion Fund (CF)
+----------------------
+
+****
+
+
+
+<table>
+<col width="50%" />
+<col width="50%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>MFF heading</p></td>
+<td align="left"><p>Smart and Inclusive Growth - Economic, Social and Territorial Cohesion</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Funds</p></td>
+<td align="left"><p>Cohesion Fund</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Policy supported</p></td>
+<td align="left"><p>Regional and Urban Policy</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>2014-2020 MFF ceiling</p></td>
+<td align="left"><p>€63 399 million<sup><a href="#ftnt19">[19]</a></sup></p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Managing Member States</p></td>
+<td align="left"><p>Bulgaria, Croatia, Cyprus, Czech Republic, Estonia, Greece, Hungary, Latvia, Lithuania, Malta, Poland, Portugal, Romania, Slovakia, Slovenia</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Managing Directorate-General</p></td>
+<td align="left"><p>Directorate-General for Regional and Urban Policy (DG REGIO)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Website</p></td>
+<td align="left"><p><a href="http://ec.europa.eu/regional_policy/en/funding/cohesion-fund/">http://ec.europa.eu/regional_policy/en/funding/cohesion-fund/</a></p></td>
+</tr>
+</tbody>
+</table>
+
+The aim of the CF is to reduce economic and social disparities and to promote sustainable development in Member States whose GNI per inhabitant is less than 90% of the EU average. The funding goes to activities supporting the establishment of trans-European transport networks or to projects related to energy or transport as long as they clearly benefit the environment.
+
+Beneficiaries: 
+
+-   **For funds under shared management**, DG REGIO manages a website with a map providing direct links to lists of beneficiaries provided by EU Member States, but the page does not make a clear distinction between the beneficiaries of spending under the CF or under the European Regional Development Fund (ERDF): [http://ec.europa.eu/regional\_policy/en/atlas/beneficiaries/](http://ec.europa.eu/regional_policy/en/atlas/beneficiaries/)
+-   **For funds under centralised direct management**, a list of beneficiaries until 2013 is available in the FTS, under the heading “Cohesion Funds”: [http://ec.europa.eu/budget/fts/index\_en.htm](http://ec.europa.eu/budget/fts/index_en.htm)
+
+The European Agricultural Fund for Rural Development (EAFRD)
+------------------------------------------------------------
+
+****
+
+
+
+<table>
+<col width="50%" />
+<col width="50%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>MFF heading</p></td>
+<td align="left"><p>Sustainable Growth: Natural Resources</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p><strong>Funds</strong></p></td>
+<td align="left"><p>European Agricultural Fund for Rural Development (EAFRD)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Policy supported</p></td>
+<td align="left"><p>Agriculture and Rural Development</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>2014-2020 MFF ceiling</p></td>
+<td align="left"><p>€95 577.052 million</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Managing Member States</p></td>
+<td align="left"><p>EU28</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Managing Directorate-General</p></td>
+<td align="left"><p>Directorate-General for Agriculture and Rural Development (DG AGRI)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Website</p></td>
+<td align="left"><p><a href="http://ec.europa.eu/agriculture/cap-funding/index_en.htm">http://ec.europa.eu/agriculture/cap-funding/index_en.htm</a></p></td>
+</tr>
+</tbody>
+</table>
+
+The EAFRD is one of the two EU funds which finance the Common Agricultural Policy (CAP), the other one being the European Agricultural Guarantee Fund (EAGF), which is not an ESIF.[^ftnt20] The CAP, the agricultural policy of the EU, was established in 1962 and last reformed in 2013 to focus on 3 priorities: (i) viable food production; (ii) sustainable management of natural resources; (iii) balanced development of rural areas throughout the EU. In this context, the EAFRD focuses on the EU’s contribution to rural development programmes and is implemented through shared management with EU Member States.
+
+The overall CAP budget has decreased over the years, but still represents a significant share of the EU budget: from 71% in 1984 to 39% in 2013.
+
+Beneficiaries:
+
+-   **For funds under shared management,**DG AGRI manages a website with links to Member States websites providing information on beneficiaries of CAP payments (although there is no clear distinction between beneficiaries of the EAFRD and the EAGF): [http://ec.europa.eu/agriculture/cap-funding/beneficiaries/shared/index\_en.htm](http://ec.europa.eu/agriculture/cap-funding/beneficiaries/shared/index_en.htm)
+-   **For funds under centralised direct management,**a list of beneficiaries until 2013 is available in the FTS: [http://ec.europa.eu/budget/fts/index\_en.htm](http://ec.europa.eu/budget/fts/index_en.htm)
+
+The European Maritime and Fisheries Fund (EMFF)
+-----------------------------------------------
+
+****
+
+
+
+<table>
+<col width="50%" />
+<col width="50%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>MFF heading</p></td>
+<td align="left"><p>Sustainable Growth: Natural Resources</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p><strong>Funds</strong></p></td>
+<td align="left"><p>European Maritime and Fisheries Fund (EMFF). Previously called the Financial Instrument for Fisheries Guidance (FIFG - for the period 1996-2006) and the European Fisheries Fund (EFF - for the period 2007-2013)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Policy supported</p></td>
+<td align="left"><p>Maritime Affairs and Fisheries</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>2014-2020 MFF ceiling</p></td>
+<td align="left"><p>€7 404.84 million</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Managing Member States</p></td>
+<td align="left"><p>EU28, Luxembourg excepted</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Managing Directorate-General</p></td>
+<td align="left"><p>Directorate-General for Maritime Affairs and Fisheries (DG MARE)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Website</p></td>
+<td align="left"><p><a href="http://ec.europa.eu/fisheries/cfp/emff/index_en.htm">http://ec.europa.eu/fisheries/cfp/emff/index_en.htm</a></p></td>
+</tr>
+</tbody>
+</table>
+
+The EMFF supports the implementation of the European Common Fisheries Policy (CFP) and of the EU Integrated Maritime Policy (IMP) for the period 2014-2020. The objectives of the EMFF are: i) to help fishermen in the transition to sustainable fishing; ii) to support coastal communities in diversifying their economies; iii) to finance projects that create new jobs and improve quality of life along European coasts. In order to reach this objective, 11% of the fund is managed directly by the European Commission to support EU-wide objectives in maritime and coastal affairs, whereas the remaining 89% are managed by EU Member States on the basis of co-financing. In the last case, each country is allocated a share of the total fund budget, based on the size of its fishing industry - this makes Spain the biggest beneficiary of the EMFF.
+
+A part of the MFF ceiling dedicated to the EMFF (€900 million) has been attributed to the establishment and management of sustainable fisheries partnership agreements with third countries. Very often criticised, these partnership agreements obey to different financial rules than the general rules applicable to the EMFF (see: [http://ec.europa.eu/fisheries/cfp/international/agreements/index\_en.htm](http://ec.europa.eu/fisheries/cfp/international/agreements/index_en.htm)).
+
+Beneficiaries:
+
+-   **For funds under shared management,**DG MARE maintains a website with links to the websites of Member States providing information on beneficiaries of EMFF payments (although, for some countries, the information presented seems to be outdated): [http://ec.europa.eu/fisheries/contracts\_and\_funding/the\_european\_transparency\_initiative/index\_en.htm](http://ec.europa.eu/fisheries/contracts_and_funding/the_european_transparency_initiative/index_en.htm) 
+-   **For funds under centralised direct management,** a list of beneficiaries until 2013 is available in the FTS: [http://ec.europa.eu/budget/fts/index\_en.htm](http://ec.europa.eu/budget/fts/index_en.htm)
+
+The European Regional Development Fund (ERDF)
+---------------------------------------------
+
+
+
+<table>
+<col width="50%" />
+<col width="50%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>MFF heading</p></td>
+<td align="left"><p>Smart and Inclusive Growth - Economic, Social and Territorial Cohesion</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p><strong>Funds</strong></p></td>
+<td align="left"><p>Less-developed regions; More developed regions; Outermost and sparsely populated regions; Transition regions; Territorial cooperation.</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Policy supported</p></td>
+<td align="left"><p>Regional and Urban Policy</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>2014-2020 MFF ceiling</p></td>
+<td align="left"><p>€187.4 million</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Managing Member States</p></td>
+<td align="left"><p>EU28</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Managing Directorate-General</p></td>
+<td align="left"><p>Directorate-General for Regional and Urban Policy (DG REGIO)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Website</p></td>
+<td align="left"><p><a href="http://ec.europa.eu/regional_policy/en/funding/erdf/">http://ec.europa.eu/regional_policy/en/funding/erdf/</a> </p></td>
+</tr>
+</tbody>
+</table>
+
+The ERDF was established in 1975. For the 2014-2020 period, its objectives are to strengthen economic and social cohesion in the EU by correcting imbalances between its regions. To do so, it focuses on several key priority areas: (i) innovation and research; (ii) the digital agenda; (iii) support for small and medium-sized enterprises; (iv) the low-carbon economy.
+
+EU regions are classified into several categories (see [the map presenting this classification](http://ec.europa.eu/regional_policy/en/policy/how/is-my-region-covered/)). Depending on this classification, different rules apply:
+
+-   In more developed regions, at least 80% of the funds must focus on at least two of the key priorities;
+-   This is 60% of the funds in transition regions;
+-   And 50% in less-developed regions.
+
+Implementation of the ERDF is made at the regional and national levels through the principles of shared management and of co-financing. The Commission retains a supervisory role.
+
+The ERDF is not established as such in the MFF and in the annual budget regulations. The money channeled through this fund comes from different budget lines, such as “Less-developed regions”, “More developed regions” or “Transition regions”. These budget lines are also used to finance other funds such as the European Social Fund (ESF), making it difficult for the public to determine the exact amount of money invested in the ERDF.
+
+Beneficiaries:
+
+-   **For funds under shared management,** DG REGIO maintains a website with links to the websites of Member States providing information on beneficiaries of ERDF payments (although the data is mixed up with the one provided for the Cohesion Fund): [http://ec.europa.eu/regional\_policy/en/atlas/beneficiaries/](http://ec.europa.eu/regional_policy/en/atlas/beneficiaries/)
+-   **For funds under centralised direct management,**a list of beneficiaries until 2013 is available in the FTS: [http://ec.europa.eu/budget/fts/index\_en.htm](http://ec.europa.eu/budget/fts/index_en.htm)
+
+The European Social Fund (ESF)
+------------------------------
+
+
+
+<table>
+<col width="50%" />
+<col width="50%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>MFF heading</p></td>
+<td align="left"><p>Smart and Inclusive Growth - Economic, Social and Territorial Cohesion</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p><strong>Funds</strong></p></td>
+<td align="left"><p>Less-developed regions; More-developed regions; Transition regions; Youth Employment Initiative</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Policy supported</p></td>
+<td align="left"><p>Employment, Social Affairs and Inclusion</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>2014-2020 MFF ceiling</p></td>
+<td align="left"><p>€86.4 million</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Managing Member States</p></td>
+<td align="left"><p>EU28</p></td>
+</tr>
+<tr class="even">
+<td align="left"><p>Managing Directorate-General</p></td>
+<td align="left"><p>Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL)</p></td>
+</tr>
+<tr class="odd">
+<td align="left"><p>Website</p></td>
+<td align="left"><p><a href="http://ec.europa.eu/esf/home.jsp?langId=en">http://ec.europa.eu/esf/home.jsp?langId=en</a></p></td>
+</tr>
+</tbody>
+</table>
+
+The ESF is the EU’s main instrument for supporting jobs, helping people to get better jobs and ensuring fairer job opportunities. For the 2014-2020 period, priorities are to boost the adaptability of workers; improving access to employment and helping people from disadvantaged groups to get jobs.
+
+The ESF is implemented through the system of shared management, together with EU Member States, and according to the co-financing principle. Co-financing rates vary between 50% and 85% of the total project costs depending on the relative wealth of the region (see [the map](http://ec.europa.eu/esf/main.jsp?catId=525&langId=en) presenting the classification of the EU regions into different categories).
+
+The ESF is not established as such in the MFF and in the annual budget regulations. The money channeled through this fund comes from different budget lines, such as “Less-developed regions”, “More developed regions” or “Transition regions”. These budget lines are also used to finance other funds such as the European Regional Development Fund (ERDF), making it difficult for the public to determine the exact amount of money invested in the ESF. In addition, and even if it is included under a different budget heading, the Youth Employment Initiative appears to be managed under the framework of the ESF.
+
+Beneficiaries:
+
+-   **For funds under shared management,** DG EMPL maintains a website with links to the websites of Member States providing information on beneficiaries of ESF payments: [http://ec.europa.eu/esf/main.jsp?catId=45&langId=en](http://ec.europa.eu/esf/main.jsp?catId=45&langId=en) 
+-   **For funds under centralised direct management,**a list of beneficiaries until 2013 is available in the FTS: [http://ec.europa.eu/budget/fts/index\_en.htm](http://ec.europa.eu/budget/fts/index_en.htm)
+
+****
+
+* * * * *
+
+****
+
+**Figure 5: Share of the EU budget per headings and funds (2015 Appropriation Commitments)**
+
+****
+
+![](../img/image02.png)
+
+Source of the data: [European Commission](http://ec.europa.eu/budget/figures/interactive/index_en.cfm) -  Made with Many Eyes (IBM)
+
+****
+
+* * * * *
+
+****
+
+**Figure 4: Members States contributors vs. beneficiaries (budget 2013)**
+
+![](../img/image04.png)
+
+Source of the data: [European Commission](http://ec.europa.eu/budget/figures/interactive/index_en.cfm) - Made with raw.densitydesign.org
+
+* * * * *
+
+All Funds
+---------
+
+{% for fund in site.eu_funds %}
+[{{fund.title}}]({{ fund.url }}) *{{ fund.objectives }}*
+{% endfor %}
+
+- [Return to Beginning](../)
+- [Prev: Who Manages and Spends the Money, and How?](../expenditure/)
+- [Next: How Transparent is the EU Budget?](../transparency/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/glossary.md

@@ -0,0 +1,65 @@
+---
+title: Glossary
+---
+
+Most of these definitions are taken from the [EU Glossary](http://ec.europa.eu/budget/explained/glossary/glossary_en.cfm) and from the publication [How the European Union works.](http://bookshop.europa.eu/en/how-the-european-union-works-pbNA3212336/)
+
+****
+
+**Budget**         Annual financial plan, drawn up according to budgetary principles, that provides forecasts and authorises, for each financial year, an estimate of future costs and revenue and expenditures and their detailed description and justification. In the EU, it takes the form of a **regulation**.
+
+**Budget Data        **Information used in the budgetary cycle - including planning, revenues and expenditures.
+
+**Ceiling        **Limits of expenditure or revenue fixed by law or by **regulation** such as in the own resources decision or in the** multiannual financial framework**.
+
+**Commitment        **Legal pledges to provide finance, provided that certain conditions are fulfilled.
+
+**Consent Procedure        **Through this procedure, the** Council** has to obtain the **European Parliament**’s assent before certain decisions are taken. The Parliament may accept or reject the **Council** proposal but cannot amend it.
+
+**Council        **The institution where ministers of EU Member States meet to discuss EU matters, take decisions and pass laws. Which ministers attend which council meeting depends on the subject on the agenda. The Council adopts the **budget** jointly with the **European Parliament**. It should not be confused with the European Council, which brings together Heads of States or Governments.
+
+**Discharge (Granting Discharge)        **The decision by which the **European Parliament**, on the basis of the recommendation from the **Council** and the declaration of assurance provided by the **European Court of Auditors**, closes the life of an annual budget exercise.
+
+**European Commission        **Politically independent executive arm of the European Union that proposes legislation, policies and programmes of actions and is responsible for implementing the decisions of the **European** **Parliament** and of the **Council**.
+
+**European Court of Auditors        **Independent external audit institution of the European Union. It checks that the EU **budget** is correctly implemented.
+
+**European Parliament        **Directly elected legislative arm of the European Union. It shares authority with the **Council** over the budget. At the end of the budget procedure, it adopts or rejects the **budget** in its entirety (“**discharge**”).
+
+**Financial Transparency System        **Database of beneficiaries of **grants** and other aids paid out each year by the **European** **Commission**.
+
+**Fiscal Transparency        **The International Monetary Fund (IMF) defines fiscal transparency as the comprehensiveness, clarity, reliability, timeliness, and relevance of public reporting on the past, present, and future state of public finances.
+
+**Grants        **Direct financial contributions, by way of donations, from the **budget** in order to finance either an action intended to help achieve an objective part of a EU policy or the functioning of a body which pursues an aim of general European interest or has an objective forming part of a EU policy.
+
+**Headings        **In the Multiannual Financial Framework, they are groups of EU activities in broad categories of expenditure.
+
+****
+
+**Multiannual Financial Framework        **Multiannual spending plan that translates into financial terms the EU policy priorities. It sets (a) limits on EU expenditure over a fixed period and thus imposes budgetary discipline; (b) annual maximum amounts (**ceilings**) of **commitments** for the main categories of expenditures (**headings**) and an overall **payments** ceilings.
+
+**Open Data        **Data is considered to be open if anyone can freely use, reuse and redistribute them, for any purpose, without restrictions (as per OpenDefinition.org). This includes being both technically open - the data can be downloaded online in open formats and read by software - and legally open - users have a legal right to reuse the data.
+
+**Open Budget Data        **Public financial information used in the budget cycle that is freely available in a machine-­readable format to use, modify and share (as per opendefinition.org).
+
+**Open Spending Data        **Information about public expenditure that is freely available in a machine­-readable format to use, modify and share (as per opendefinition.org)
+
+**Open Fiscal Data**        Information about public finance that is freely available in a machine-­readable format to use, modify and share (as per opendefinition.org).
+
+**Ordinary Legislative Procedure        **Main legislative procedure by which directives and **regulations** are adopted. It gives an equal weight to the **European Parliament** and to the **Council**in the decision procedure.
+
+**Own Resources        **Revenue flowing automatically to the European Union **budget**, pursuant to the Treaties and implementing legislation, without the need for any subsequent decision by national authorities.
+
+**Payment        **Cash or bank transfer to the beneficiaries.
+
+**Regulation**        Normative act, of general application, binding in its entirety and directly applicable in all EU Member States.
+
+**Spending Data        **Spending data is information about public expenditures - and crucially includes transaction level data, e.g. data about individual financial transactions. Ideally spending data should contain information on amount, payer, beneficiary, date and purpose of the transaction (e.g. a project or service).
+
+**Sugar Levies        **Mainly customs duties on imports of certain agricultural products from non-EU countries as well as levies paid by producers on sugar, isoglucose and insulin syrup.
+
+- [Return to Beginning](../)
+- [Prev: Acronyms](../acronyms/)
+- [Next: The EU Budget at a Glance](../overview/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/index.md

@@ -0,0 +1,118 @@
+---
+title: "Where Does Europe’s Money Go?"
+authors:
+- Elisabeth Druel
+- Pierre Chrzanowski
+- Rufus Pollock
+- Jonathan Gray
+redirect_from:
+- /research/eu/
+---
+
+## A Guide to EU Budget Data Sources
+
+This report was researched and written by **Elisabeth Druel** and **Pierre Chrzanowski** on behalf of **Open Knowledge**, with input and guidance from **Rufus Pollock** and **Jonathan Gray**. Support was provided by the **Adessium Foundation**.
+
+**Open Knowledge** is a global civil society organisation dedicated to opening up public information, research and culture to benefit the lives of citizens around the world. Find out more at: [okfn.org](https://okfn.org/).
+
+**Adessium Foundation** aspires to a society that encourages people to live in harmony with each other and with their environments. The Foundation works to achieve a balanced society characterized by integrity, a balance between people and nature, and social harmony. Find out more at: [adessium.org](http://www.adessium.org/).
+
+Thanks to the following people who provided input, feedback and support at various stages:
+
+-   **Brigitte Alfter**, Europe Editor, JournalismFund.eu and Lecturer, Roskilde University
+-   **Caelainn Barr**, Data Journalist, The Guardian
+-   **Cynthia O’Murchu**, Reporter, Financial Times
+-   **Nick Aiossa**, EU Policy Officer, Transparency International EU Office
+
+## Executive Summary
+
+<table>
+<col width="100%" />
+<tbody>
+<tr class="odd">
+<td align="left"><p>The EU Budget in Numbers</p>
+<p></p>
+<ul>
+<li>The EU has committed to spending over €959 988 million over the period 2014-2020.</li>
+</ul>
+<ul>
+<li>The EU budget is divided into more than 80 funds and programmes.</li>
+<li>Five of these funds, the European Structural and Investments Funds (ESIF), represent 33% of the total budget.</li>
+<li>Around 6% of the budget goes to the administration of the European institutions, whereas around 94% is allocated to various European programmes.</li>
+<li>There are more than 100 authorities managing EU funds. 20% of these are managed directly by the European Commission. The remaining 80% are managed by other authorities.</li>
+</ul></td>
+</tr>
+</tbody>
+</table>
+
+Recent polls show that European citizens hold the EU to be inaccessible due to its complexity and lack of transparency.[^ftnt1] The lack of transparency and democratic accountability in European institutions is widely characterised as a “[democratic deficit](http://europa.eu/legislation_summaries/glossary/democratic_deficit_en.htm)”, a phrase which has become part of the EU’s official glossary. This “democratic deficit” is particularly apparent when it comes to EU public finances.
+
+This guide aims to help civil society organisations (CSOs), journalists and others to navigate the vast landscape of documents and datasets about the EU’s fiscal affairs. In doing so, our objective is to support more evidence-based journalism and advocacy, and - in the longer term - to contribute towards the transparency, public understanding and democratic accountability of EU public finances.
+
+The level of transparency about EU public money is highly variable, and is largely dependent on which authority is responsible for managing and disbursing a given fund. Funds which are exemplary for their transparency exist alongside funds which are effectively dark. For example, on one hand, the Financial Transparency System (FTS) set up by the European Commission provides access to granular open data about spending, but only concerns about 20% of total spending. On the other hand, information about public money spent by EU Members States is often unavailable to the public.
+
+The European Union has now entered into a new budget framework for 2014-2020. This is accompanied by a new financial regulation, defining new funds and new transparency rules. However, as we shall see in this report, many barriers still remain. In addition to providing an overview of key funds and programmes, we also conclude with some analysis and recommendations for further work in this area, which are summarised below.
+
+
+
+<table>
+<col width="100%" />
+<tbody>
+<tr class="odd">
+<td align="left"><h2>Summary of Recommendations</h2>
+<p></p>
+<ol>
+<li><strong>Establish a single central point of reference</strong> for data and documents about EU revenue, budgeting and expenditure and ensure all the information is up to date  at this domain (e.g. at a website such as <a href="http://www.google.com/url?q=http%3A%2F%2Fec.europa.eu%2Fbudget&amp;sa=D&amp;sntz=1&amp;usg=AFQjCNFWcyA9aOiTTUegt8YVevYXpnkn0Q">ec.europa.eu/budget</a>). At the same time, ensure all EU budget data are available from the EU open data portal as open data.</li>
+</ol>
+<p></p>
+<ol start="2">
+<li><strong>C</strong><strong>reate an open dataset with key details about each EU fund</strong>, including name of the fund, heading, policy, type of management, implementing authorities, link to information on beneficiaries, link to legal basis in Eur-Lex and link to regulation in Eur-Lex.</li>
+</ol>
+<p></p>
+<ol start="3">
+<li><strong>Extend the Financial Transparency System to all EU funds</strong> by integrating or federating detailed data expenditures from Members States, non-EU Members and international organisations. Data on beneficiaries should include, when relevant, a unique European identifier of company, and when the project is co-financed, the exact amount of EU funding received and the total amount of the project.</li>
+</ol>
+<p></p>
+<ol start="4">
+<li><strong>Clarify and harmonise the legal framework regarding transparency rules</strong> for the beneficiaries of EU funds.</li>
+</ol>
+<p></p>
+<ol start="5">
+<li><strong>Support and strengthen funding for civil society groups and journalists</strong> working on EU public finances.</li>
+</ol>
+<p></p>
+<ol start="6">
+<li><strong>Conduct a more detailed</strong> <strong>assessment</strong><strong> of beneficiary data availability</strong> for all EU funds and for all implementing authorities - e.g., through a dedicated “open data audit”.</li>
+</ol>
+<p></p>
+<ol start="7">
+<li><strong>Build a stronger central base of evidence about the uses and users of EU fiscal data</strong> - including data projects, investigative journalism projects and data users in the media and civil society.</li>
+</ol>
+<p></p></td>
+</tr>
+</tbody>
+</table>
+
+## Table of Contents
+
+- [The EU Budget at a Glance](./overview/)
+- [How is the EU Budget Set Up?](./budget/)
+- [Where Does the Money Come From?](./revenue/)
+- [Who Manages and Spends the Money, and How?](./expenditure/)
+- [Where Does the Money Go?](./funds/)
+- [How Transparent is the EU Budget?](./transparency/)
+- [Journalistic Investigations and Transparency Projects to Follow the Money](./research/)
+- [Recommendations and Next Steps](./recommendations/)
+- [Annex - The Open Data Audit of EU Funds](./datasets/)
+- [Annex - Legal Basis for the Establishment of the EU budget](./legal-basis/)
+- [References](./references/)
+- [Acronyms](./acronyms/)
+- [Glossary](./glossary/)
+
+## Download the PDF version
+
+<a href="./pdf/WhereDoesEuropesMoneyGo.pdf">
+<img src="./img/cover.png">
+</a>
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/legal-basis.md

@@ -0,0 +1,25 @@
+---
+title: Annex - Legal Basis for the Establishment of the EU budget
+---
+
+**Treaty on the Functioning of the European Union (TFEU), Title II - Financial Provisions - Articles 310 to 325:**[^ftnt33]
+
+Article 310: *“The Union’s annual budget shall be established by the European Parliament and the Council (...). The revenue and expenditure shown in the budget shall be in balance”.*
+
+Article 311: *“The Union shall provide itself with the means necessary to attain its objectives and carry through its policies”.*
+
+Article 312: *“The multiannual financial framework shall ensure that Union expenditure develops in an orderly manner and within the limits of its own resources. It shall be established for a period of at least five years. The annual budget of the Union shall comply with the multiannual financial framework”.*
+
+Article 314: *“The European Parliament and the Council, acting in accordance with a special legislative procedure, shall establish the Union’s annual budget (...)”.*
+
+Article 318: *“The Commission shall submit annually to the European Parliament and to the Council the accounts of the preceding financial year relating to the implementation of the budget”.*
+
+Article 319: *“The European Parliament, acting on a recommendation from the Council, shall give discharge to the Commission in respect of the implementation of the budget”.*
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: Annex - The Open Data Audit of EU Funds](../datasets/)
+- [Next: References](../references/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/overview.md

@@ -0,0 +1,15 @@
+---
+title: The EU Budget at a Glance
+---
+
+![](../img/image07.png)
+
+Source: European Commission
+
+[SVG version available here](https://drive.google.com/open?id=0BxUOTiQczyxHY2ZBV0daZ1huMnc&authuser=0) 
+
+- [Return to Beginning](../)
+- [Prev: Glossary](../glossary/)
+- [Next: How is the EU Budget Set Up?](../budget/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/recommendations.md

@@ -0,0 +1,31 @@
+---
+title: Recommendations and Next Steps
+---
+
+As this report has evidenced, the level of transparency of EU funds is not consistent and depends on who manages the fund. **The result is a two-sided system where best practices in terms of transparency stand alongside closed data.** On one hand, the FTS of the European Commission delivers access to real expenditures information following open data standards, but it only concerns about 20% of EU spending. On the other hand, data on money spent through Members States is often unavailable to the public, and if available, not accessible in different languages. Furthermore, the release of EU Budget data as open data is still not a default practice.
+
+The European Union has now entered a new MFF for 2014 -2020, with the adoption of a new financial regulation, defining new funds and transparency rules. However, structural barriers still remain, preventing an effective transparency of all EU funds and public accountability of all managing authorities.
+
+Therefore, based on the EU budget mechanisms analysis, related transparency rules, and a preliminary assessment of effective EU budget and funds data availability, this report recommend the following actions to improve EU budget transparency and accountability:
+
+1.  **Establish a single central point of reference**** ****for EU Budget information** and ensure all up to date documents and data are linked under this domain (e.g. at a website such as [ec.europa.eu/budget](http://ec.europa.eu/budget)). In the same time, **e****nsure all EU Budget data are available from the EU open data portal as open data.**
+
+1.  **Set up a unique dataset of EU funds available as open data to improve the overall transparency of the EU budget.**The dataset could include the following information: name of the fund, heading, policy, type of management, implementing authorities, link to information on beneficiaries, link to legal basis in Eur-Lex and link to regulation in Eur-Lex.
+
+1.  **Extend the Financial Transparency System to all EU funds by centralising or federating detailed data expenditures from Members States, non-EU Members and international organisations.** Data on beneficiaries should include, when relevant, a unique European identifier of company, and when the project is co-financed, the exact amount of EU funding received, the total amount of the project and all other co-financing sources.
+
+1.  **Clarify the legal framework regarding the rules applicable to the transparency of beneficiaries of EU funds and ensure its consistency across the European Union.** Although this report did not conduct a comprehensive legal review of EU budget transparency, it has found that: (i) the differences of transparency rules applying to the beneficiaries of funds managed under centralised and shared management are preventing the establishment of a common transparency standard for all EU expenditures. This should concern also the publication, by the European institutions and in open data format, of more detailed expenditure information under the “Administration” heading; (ii) the differences between data protection law and effective enforcement within each Member State are also preventing a high level of EU funds transparency across Europe; (iii) the interpretation of the principle of proportionality regarding data protection is still subject to discussion; (iv) the transparency rules adopted by third countries and international organisations managing EU funds may also differ from the ones adopted by the EU and its Member States and prevent the establishment of a common transparency framework.
+
+1.  **Support and strengthen civil society and journalists capacities on EU funds, together with the development of common data and investigation tools.** Newsrooms are too often reticent to launch long data investigations without the guarantee that these will pay off with good stories. In the wake of JournalismFund.eu, there is a need to support funding for data skills and relevant tools to develop data journalism and investigation projects related to EU funds.[^ftnt32] This should include not only literacy with datasets, but also literacy with data infrastructures, institutions and policies. Projects of interest in this area may include issues such as contracts/tenders financed or co-financed through the EU budget, in order to determine what are the regulations governing their functioning, what level of information is currently open to public scrutiny and what are the loopholes which may eventually be corrected in this system both to facilitate access to the data and to prevent fraud or misuse of the funds.
+
+1.  **Conduct an assessment of EU funds beneficiary data availability (Open Data audit of EU funds).**** **Data on EU funds beneficiaries are spread into multiple data system managed by various implementing authorities such as EU institutions, Member States and international organisations. The open data audit of EU funds seeks to map all this data management system, the data they contain (from which EU funds), and their level of openness against a specific set of criteria such as the ones used in the [Open Data Index](http://index.okfn.org/). The Open Data audit of EU funds would be an essential tool for any further investigation into EU funds data as well as a powerful benchmark for any advocacy effort towards greater transparency in Europe and to assist official accountability initiatives and public institutions in identifying transparency gaps.
+
+1.  **Build a stronger central base of evidence about the uses and users of EU fiscal data - including data projects, investigative journalism projects and data users in the media and civil society.**As well as contributing to a richer empirical picture of how fiscal data is actually used, we would like to see this work informing the development of a framework to highlight how information about EU public finances can be used to improve democratic accountability - including the different “accountability paths” through which this information can be put to work in society.
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: Journalistic Investigations and Transparency Projects to Follow the Money](../research/)
+- [Next: Annex - The Open Data Audit of EU Funds](../datasets/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/references.md

@@ -0,0 +1,36 @@
+---
+title: References
+---
+
+European Commission (2013). Multiannual Financial Framework 2014-2020 and EU Budget 2014 in a nutshell. Available at: [http://ec.europa.eu/budget/news/article\_en.cfm?id=201401281520](http://ec.europa.eu/budget/news/article_en.cfm?id=201401281520) 
+
+European Commission (2013). The European Union Explained — How the European Union Works. Available at: [http://bookshop.europa.eu/en/how-the-european-union-works-pbNA0414810/](http://www.google.com/url?q=http%3A%2F%2Fbookshop.europa.eu%2Fen%2Fhow-the-european-union-works-pbNA0414810%2F&sa=D&sntz=1&usg=AFQjCNGNF0rng49wWTbq3BnqZdFY8bfC4Q) 
+
+Official Journal of the European Union (2013). Council Regulation (EU, EURATOM) No 1311/2013. Available at: [http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0884:0891:EN:PDF](http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2013:347:0884:0891:EN:PDF) 
+
+European Parliament (2012). Deterrence of fraud with EU funds through investigative journalism in EU-27, 2012. Available at: [http://www.journalismfund.eu/sites/default/files/EP%20Study%20-%20Deterrence%20of%20fraud.pdf](http://www.journalismfund.eu/sites/default/files/EP%20Study%20-%20Deterrence%20of%20fraud.pdf) 
+
+## Official sources of EU budget data
+
+For the source of EU funds beneficiaries, please refer to the [Open Data audit of EU funds](https://docs.google.com/spreadsheets/d/1tkKxRlkW60-ylxdvxGkMlMq8BS4SRPR4QoEd72qgFwQ/edit#gid=2028897927)
+
+**EU Budget 2006 - 2013 (revenue and expenditures)**
+
+[http://ec.europa.eu/budget/revexp/revenue\_and\_expenditure\_files/data/revenue\_and\_expenditure\_en.xls](http://ec.europa.eu/budget/revexp/revenue_and_expenditure_files/data/revenue_and_expenditure_en.xls)
+
+**Multiannual Financial Framework 2014 - 2010**
+
+[http://ec.europa.eu/budget/mff/lib/data/MFF2014-2020.xls](http://ec.europa.eu/budget/mff/lib/data/MFF2014-2020.xls)
+
+**Cohesion Policy Data**
+
+Information,** available as open data**, about the investments made and the results obtained under the Cohesion Policy (as well as contextual data on the socio-economic situation in each EU country).
+
+[h](https://cohesiondata.ec.europa.eu/)[ttps://cohesiondata.ec.europa.eu/](https://cohesiondata.ec.europa.eu/)
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: Annex - Legal Basis for the Establishment of the EU budget](../legal-basis/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/research.md

@@ -0,0 +1,97 @@
+---
+title: Journalistic Investigations and Transparency Projects to Follow the Money
+---
+
+There is a growing network of investigative journalists in Europe, supported by initiatives such as the JournalismFund.eu, who are acquiring skills to work on complex data projects. In parallel, a number of data-savvy analysts, activists and developers, are supporting investigations efforts by extracting (scraping), gathering and releasing EU budget data in a more useful way and developing dedicated tools and platforms. Those practices are often described as data journalism.[^ftnt27] Investigations can also be conducted or initiated by EU public authorities which have the mandate to control the EU Budget. The following sections present a brief and illustrative overview of a selection of recent work.[^ftnt28]
+
+Investigations into EU Funds
+----------------------------
+
+**EUobserver**
+
+The EUobserver is a non-profit independent online newspaper established in Brussels. It regularly covers [EU funds attribution, fraud and misuses](https://euobserver.com/search?q=EU+funds). In 2013, the EUobserver investigated [EU funds attribution to regions](https://euobserver.com/regions/119867). The result was that the poorer and more mismanaged a region was, the less it was profiting from EU funds, therefore raising the point that there is a relationship between administrative capacity and good and effective use of funds.
+
+**EU Transparency**
+
+EU Transparency is a non-profit organisation based in the UK and working to improve transparency of the European Institutions. EU transparency was behind the project FishSubsidy (see next section) and host of the Farmsubsidy.org project for some years.
+
+**European Research Centre for Anti-Corruption and State-Building**
+
+The European Research Centre for Anti-Corruption and State-Building (ERCAS) is an academic research center located at the [Hertie School of Governance](http://www.hertie-school.org/) in Berlin, Germany.** **ERCAS** **has** **notably established the [anti-corruption project database](http://www.againstcorruption.eu/?post_type=projects), where anti-corruption initiatives are documented and promoted through a database of projects, impact analysis and diagnosis reports. ERCAS also investigates itself EU funds abuse and [conducted a project in Bulgaria](http://www.againstcorruption.eu/projects/effective-investigation-of-european-union-funds-abuse-in-bulgaria/).
+
+**Financial Times**
+
+The online version of the Financial Times has [a dedicated section](http://www.ft.com/intl/topics/themes/EU_Structural_Funds) on investigations and articles on the EU Structural Funds where it reports about EU funds management issues, frauds and funds misuse. The last article raised the issue of UK local councils, London excepted, not being allowed to independently manage their share of the ESIF funds. The Financial Times has also worked with the Bureau of Investigative Journalism to make EU Structural Funds more public. (see The Bureau of Investigative Journalism and data project description in next section).
+
+**JournalismFund.eu**
+
+The [JournalismFund.eu](http://journalismfund.eu/) was founded in 2008 on the model of the [Belgian Pascal Decroos Fund](http://www.fondspascaldecroos.org/en) and is the only European fund providing working grants for journalism investigation across the EU. The fund also supports the [Dataharvest - European Investigative Journalism Conference](http://www.journalismfund.eu/dataharvest-conferences) which has become the main annual conference for journalists to share best practices and story ideas related to EU data. 
+
+**The Bureau of Investigative Journalism**
+
+The Bureau of Investigative Journalism (TBIJ) is a non-for-profit news organisation based in London. It has worked with the Financial Times on the [EuFunds project](https://www.thebureauinvestigates.com/category/projects/eu-structural-funds/) to investigate EU Structural Funds. One of the main findings from that investigation is that [only 5% (54) of 1046 leaks](https://www.thebureauinvestigates.com/2012/07/07/eu-member-states-fail-to-report-corruption-but-bureau-investigation-hits-home/) relating to corruption in 2011 came from Member States authorities, raising concerns over the shared management of EU funds.
+
+**The International Consortium of Investigative Journalists**
+
+[The International Consortium of Investigative Journalists (ICIJ)](http://www.icij.org/) is a global network of about 200 investigative journalists who collaborate on investigative stories across the globe. The ICIJ was launched as a project of the Center for Public Integrity. In 2012, the ICIJ published *“*[Looting the seas II](http://www.icij.org/projects/looting-seas-ii)*”*, an in-depth investigation into EU subsidies supporting Spanish fisheries. The investigation evidences that *"decades of overfishing have left Europe’s fish stocks in peril and its fishermen in poverty. It’s an impasse paid for by EU taxpayers."* Among the key findings, it has been found that one Spanish ship owner received more than €8.2 million in subsidies even while his company or its affiliates faced more than 40 accusations of illegal fishing and US\$5 million in fines.[^ftnt29] ICIJ used various data sources for its investigation, including data from FishSubsidy.org (see section below) but also had to requests additional information to the EU and the national and regional governments in Spain.[^ftnt30]
+
+**Transparency International EU Office**
+
+As part of the global Transparency International, the Transparency International EU Office (TI-EU) focuses on EU advocacy. It promotes accountability, transparency and integrity at the EU level. TI-EU recognised that the shared management system of a large number of EU funds makes it very difficult to access data, and is therefore [calling](http://www.transparencyinternational.eu/focus_areas/eu-budget/) for all EU funds to be available on a single, citizen-friendly website.
+
+Investigations can also be conducted or initiated at the EU level by dedicated authorities.
+
+**European Anti-Fraud Office (OLAF)**
+
+[OLAF](http://ec.europa.eu/anti_fraud/index_en.htm)is  charged by the [E](http://en.wikipedia.org/wiki/European_Union)U of the protection of the financial interests of the EU budget. It investigates fraud, corruption and serious misconduct within the European institutions, and develops an anti-fraud policy for the European Commission. OLAF's mandate covers all EU expenditures and it cooperates with national authorities for funds under shared management. OLAF publishes an annual report containing statistics on fraud and examples of cases investigated. According to [the 2013 repor](http://ec.europa.eu/anti_fraud/about-us/reports/olaf-report/index_en.htm)t, the main amounts of fraud were in structural funds (€111.7 million), external aid (€100.4 million), customs and trade (€84.9 million), new financial instruments (€66.3 million) and agricultural funds (€34.4 million).
+
+**European Ombudsman**
+
+The [Ombudsman](http://www.ombudsman.europa.eu/atyourservice/whocanhelpyou.faces#/page/4) is an independent watchdog with the power to investigate EU institutions. It regularly conducts investigation into EU funds. For instance, on 21 May 2014, the European Ombudsman has launched [an investigation](http://www.ombudsman.europa.eu/en/press/release.faces/en/54420/html.bookmark)on the EU [Cohesion Policy](http://ec.europa.eu/regional_policy/what/future/index_en.cfm), following concerns that substantial funding is being used to breach the fundamental rights of EU citizens with disabilities.
+
+**The European Court of Auditors**
+
+[The European Court of Auditors](http://www.eca.europa.eu/) is the independent external auditor of the EU institutions. Its role is to check that EU funds are correctly and efficiently managed and raised and spent in accordance with the relevant rules and regulations. In particular, the court checks if projects funded through direct and shared management are in line with EU priorities.
+
+**Project ARACHNE** 
+
+[ARACHNE](http://admin.interact-eu.net/downloads/7857/Presentation_) is a project from the European Commission which aims at providing Member State authorities involved in the management of the structural funds with an operational tool to identify potentially risky projects. The application collects and process data from Members States EU-funded projects, together with financial data on companies (data retrieved from [ORBIS](http://www.bvdinfo.com/en-gb/our-products/company-information/international-products/orbis?gclid=CPq2-d_am8UCFeoSwwodZVcACg)), compliance data such as list of fraudsters (data from [World Compliance, Lexis Nexis](http://www.lexisnexis.com/risk/intl/en/)) and web mining data from news articles (data retrieved from [Moreover](http://www.moreover.com/)). The project managed by DG EMPL is already in operation Bulgaria, Czech Republic, Hungary, Poland, Portugal, Romania and Slovakia. The tool's dashboard is available to Member States authorities and the Commission. However, there are as yet no plans to make the data publicly available.
+
+Tools for EU Funds Data Investigation
+-------------------------------------
+
+**AsktheEU.org**
+
+Ask the EU is a portal managed by [Access Info Europe](http://www.access-info.org/)  which assists any citizen requesting an information from the EU institutions under the access to information right. The portal makes every request and response publicly available online, allowing for an analysis of information request on EU budget and spending. For example there are [150 requests related to “spending](http://www.asktheeu.org/en/search/spending/all?commit=Search&utf8=%E2%9C%93)[” data](http://www.asktheeu.org/en/search/spending/all?commit=Search&utf8=%E2%9C%93) and [393 requests related to “budget](http://www.asktheeu.org/en/search/budget/all?commit=Search&utf8=%E2%9C%93)[” data](http://www.asktheeu.org/en/search/budget/all?commit=Search&utf8=%E2%9C%93).
+
+**FarmSubsidy.org** 
+
+[FarmSubsidy](http://farmsubsidy.openspending.org/) seeks to obtain detailed data relating to payments and recipients of farm subsidies in every EU member state and make this data available in a way that is useful to European citizens. This is a  project of [JournalismFund.eu](http://journalismfund.eu/), a non-profit organisation. The website is powered by OpenSpending of Open Knowledge and has previously been run by EU Transparency and DICAR (Danish International Center for Analytical Reporting). The website [lists](http://farmsubsidy.openspending.org/news/media/) over 120 stories which came out from the FarmSubsidy database.
+
+**FishSubsidy.org**
+
+Similar to FarmSubsidy, FishSubsidy was a website providing detailed data related to payments and recipients of fish subsidies in every EU Member State. The website was closed down due to lack of funding. However, [related datasets are still available online](https://docs.google.com/spreadsheets/d/1NZ9tc_JN8psXXOhCHmdzsVqhvGWsE7igDf004gHgfgQ/edit?hl=en_US#gid=4) and their T[witter account](https://twitter.com/fishsubsidy) is still active. A report of the European Parliament from 2012 on investigative journalism estimated that around 40 stories came out from the FishSubsidy data.[^ftnt31] 
+
+**EuFunds**
+
+[EuFunds](http://eufunds.ftdata.co.uk/) is an initiative from the Financial Times and [The Bureau of Investigative Journalists](http://www.thebureauinvestigates.com/) (TBIJ) to collect data on beneficiaries from EU Structural Funds. The application, still online, provides a search engine to query EU structural funds data. The data were extracted between April 2010 and September 2010 and represent a snapshot of projects that have been allocated EU funds from the start of the current funding phase in 2007 onwards. The underlying database is also available as a [MySQL](http://dl.dropbox.com/u/39827429/EU_Structural_Funds/sql/euFunds.sql) and [CSV file](http://dl.dropbox.com/u/39827429/EU_Structural_Funds/csv/euFunds.csv).
+
+**OpenTED**
+
+[OpenTED](http://ted.openspending.org/) is an initiative which aims to make information from the joint European procurement system more easily accessible to journalists and researchers. The initiative was developed in the frame of the Dataharvest.eu conference in collaboration between independent journalists, developers and researchers. The dataset contains procurement data from 2013 to 2015, including contracts from EU funds.
+
+**OpenInterests.eu**
+
+[OpenInterests.eu](http://openinterests.eu/) is a catalogue of political and commercial actors related to the European Union. The project combines different sets of information including data from the FTS into a search engine, which can be used to quickly retrieve information about the activities of companies, people and institutions in a European context.
+
+**Wobbing.eu**
+
+[Wobbing.eu](http://www.google.com/url?q=http%3A%2F%2Fwobbing.eu%2F&sa=D&sntz=1&usg=AFQjCNHVQsTSB9R-DYHnv2lPmOoxuKJPgw) is a network of journalists interested in using the EU’s freedom of information legislation at national and European levels to obtain documents. The network members exchange information on developments on national and European level and assist each other with requests when necessary. Annual meetings in the Wobbing.eu network have been merged into the annual Dataharvest.eu conference.
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: How Transparent is the EU Budget?](../transparency/)
+- [Next: Recommendations and Next Steps](../recommendations/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/revenue.md

@@ -0,0 +1,40 @@
+---
+title: Where Does the Money Come From?
+---
+
+The sources of the EU revenue are called the EU’s “own resources”. A Council Decision on Own Resources, setting up the rules applying to their calculation, is usually negotiated at the same time as the MFF regulation. Resources are then provided on an annual basis.
+
+The difference with the MFF regulation is that the Council Decision has to be approved by all the Member States before it comes into force - usually through a vote in national parliaments. This explains why, as of April 2015, the Council Decision on the EU’s own resources dates back from 2007[^ftnt11] and continues to apply until the new Council Decision, adopted in May 2014 by the Council, comes into force.[^ftnt12] When the 2014 Decision will be approved by all the Member States, it will apply retroactively as of 1 January 2014.
+
+The EU’s own resources are divided into several categories:
+
+-   **C****ustoms duties on EU imports**: they arise from the application of the common customs tariffs and are collected on imports from third countries at the external borders of the EU.
+-   **A****gricultural duties and sugar levies**: they consist mainly of customs duties on imports from non-EU countries of agricultural products subject to the common organisation of the markets and levies on sugar, isoglucose and insulin syrup.
+
+Customs duties, agricultural duties and sugar levies are also referred as the **EU “traditional” own resources**.
+
+-   **The VAT-based own resources:** these are national contributions based on harmonised VAT bases for Member States.
+-   **The GNI-based own resources:** this revenue is based on an uniform call rate applied to the Gross National Income (GNI) of each Member State.
+-   **Other revenues:**it includes taxes on EU staff; contributions from non-Member States and fines imposed on companies.
+
+The collection of revenues does not apply uniformly to all EU Member States, as rather *ad-hoc* **correction mechanisms** also exist. The three main correction mechanisms are:
+
+-   **The UK rebate**: the UK is reimbursed a percentage of the difference between its contribution and what it receives back from the EU budget (as of today, 66% of the difference). To compensate this loss, all other Member States have to make an additional contribution to the budget in proportion to the share they contribute to the EU’s GNI. Although controversial, the UK rebate has been maintained by the 2014 Council Decision on the EU own resources;
+-   **Lump-sum payments**: some countries (Denmark, the Netherlands and Sweden) benefit from gross reductions in their annual GNI contributions;
+-   **Reduced VAT call rates**for Germany, the Netherlands and Sweden.[^ftnt13]
+
+As previously noted, the European Development Fund is not concerned by these rules, as it is financed directly by the Member States and has its own financial regulations.[^ftnt14]
+
+* * * * *
+
+**Figure 2: EU own resources contribution per Member States (Budget 2013)**
+
+Source of the data: [European Commission](http://ec.europa.eu/budget/figures/interactive/index_en.cfm) - Made with raw.densitydesign.org![](../img/image06.png)![](images/image06.png)
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: How is the EU Budget Set Up?](../budget/)
+- [Next: Who Manages and Spends the Money, and How?](../expenditure/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/eu/transparency.md

@@ -0,0 +1,86 @@
+---
+title: How Transparent is the EU Budget?
+---
+
+Overall Fiscal Transparency
+---------------------------
+
+Most of the data and documentation for the budget process and its implementation are available online. The Cohesion Policy even has its own [open data website](https://cohesiondata.ec.europa.eu) offering aggregated budget data. However, there is not a unique and primary access point where all budget documents and data are centralised or federated in as standardised way. Being able to obtain a joined-up picture of EU public finances is essential for democratic accountability. For example, journalists collaborating around the EU Farm subsidy project were initially unable to determine the total agricultural subsidies going to large multi-national corporations from the EU, as the data sources were scattered across hundreds of different local, regional and national government websites, in different formats and the European Commission denied an access for documents request for the centralised dataset.[^ftnt21]
+
+Indeed, even though the Commission has [a central budget section](http://ec.europa.eu/budget/index_en.cfm) on its website, this report has evidenced that is was necessary to consult many websites or webpages not referenced under the Commission website in order to understand the basic principles related to the EU budget functioning and to collect related figures.
+
+There are also some inconsistencies between the terms and figures published in the budget documents from different EU institutions websites.[^ftnt22] As a result, accessing, analysing and understanding the whole EU Budget is a complex task which hampers EU Budget transparency.
+
+One of the main issues is the missing link between the two major EU budgetary instruments (the  seven years framework and the annual budget regulation) and the funds themselves. The funds set up by the multiannual framework and the budget are implemented, directly or not, by the Directorate-Generals and Agencies of the Commission, and specific information related to these funds is therefore to be found on their respective websites. But the information and data they provide are not presented in a harmonised way, making it difficult and extremely burdensome to gather the same level of information for each of the funds. 
+
+In addition, whereas the Commission provides a unique website to link the beneficiaries of its spending under centralised direct management (the FTS), there is no such system for the vast majority of funds spent under shared management.[^ftnt23] Diversity of languages for the different websites of the Member States also adds complexity to find and analyse the information on beneficiaries. The lack of distinction from EU and other co-funding for a project is also an issue. Lastly, datasets on beneficiaries do not use a unique European identifier for legal entities. This  hampers the capacity of investigation and other data analysis across Europe.[^ftnt24]
+
+Transparency Regulations
+------------------------
+
+According to Article 15 of the TFUE, *“the Union’s institutions , bodies, offices and agencies shall conduct their work as openly as possible. (...) Any citizen of the Union, and any natural or legal person residing or having its registered office in a Member State, shall have a right to access to documents of the Union’s institutions, bodies, offices and agencies, whatever their medium, subject to the principles and conditions to be defined in accordance with this paragraph”*.
+
+To implement this principle of transparency, specific obligations were defined in successive legal instruments called “Financial regulations”. The last financial regulation, adopted in 2012, established an obligation of publication of accounts, budgets and reports through means such as the Official Journal of the European Union. There is also an obligation of publication of information on recipients of EU funds. It is stated in the financial regulation that *“where natural persons are concerned, the publication shall be limited to the name and locality of the recipient, the amount awarded and the purpose of the award”*.[^ftnt25]
+
+Therefore, this principle of transparency is not an absolute one, as it is the subject of *“principles and conditions”*. This was evidenced in 2010, when the Court of Justice of the European Union delivered [a judgment](http://curia.europa.eu/juris/document/document.jsf;jsessionid=9ea7d0f130de9fa4dec2274b4c12a176e3a592620a2c.e34KaxiLc3eQc40LaxqMbN4ObxaSe0?text=&docid=79001&pageIndex=0&doclang=en&mode=lst&dir=&occ=first&part=1&cid=583202) in which it rules that publishing personal data on the beneficiaries of funds deriving from the two financial pillars of the CAP was not proportionate when it comes to natural persons who are the beneficiaries of these funds. As a consequence, for a two-years period, the obligation to publish information on the beneficiaries of CAP payments was limited to **only** legal persons. In 2013, [a new regulation](http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32013R1306&from=en) was adopted, which re-established the obligation to publish information on the beneficiaries of CAP payments, except when it comes to small amounts of money received by natural persons (per default less than €1250 received in one year).[^ftnt26] 
+
+The 2010 ruling of the Court certainly influenced the drafting of the 2012 financial regulations. In [the Commission Delegated regulation of 29 October 2012](http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L:2012:362:FULL&from=EN), it is stated that *“as far as personal data referring to natural persons are concerned, the information published shall be removed two years after the end of the financial year in which the funds were awarded. The same shall apply to personal data referring to legal persons for whom the official title identifies one or more natural persons”*. This limitation is applicable to recipients of EU funds awarded under direct and indirect management.
+
+For each sector or fund, more specific and detailed rules can be adopted. For example, for the 5 ESIF, [the Common Provisions regulation](http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32013R1303&:PDF)states that: *“Member States or managing authorities shall, in order to ensure transparency concerning support from the  Funds, maintain a list of operations by operational programme and by fund in a spreadsheet data format, which allows data to be sorted, searched, extracted, compared and easily published on the internet, for instance in CSV or XML format. The list of operations shall be accessible through the single website or the single website portal providing a list and summary of all operational programmes in that Member State. (...) The list of operations shall be updated at least every six months”.*
+
+****
+
+Beneficiaries of EU Funds
+
+To find information on the beneficiaries of EU funds, it is necessary to look at the type of management involved, as this has a direct implication on the place where the data will be found and on which authority (the EU itself, the Member States, etc.) will manage the publication of this data.
+
+****
+
+**Centralised Direct Management**
+
+For funds awarded under centralised direct management, the financial regulation provides that the information on the beneficiary shall be published on an internet site of the EU institutions. Information shall contain the name of the recipient; the locality of the recipient; the amount awarded; the nature and purpose of the measure.
+
+Some rules and exceptions apply as regard to data privacy. Any information identifying a natural person shall be removed two years after the end of the financial year in which the funds were awarded, and the following information should not be released:
+
+-   Public procurement contracts worth under a certain amount (currently €15000);
+-   Scholarships or other direct support paid to natural person.
+
+In order to comply with the financial regulation, the European Commission has set up the [Financial Transparency System (FTS)](http://ec.europa.eu/budget/fts/index_en.htm). In this public database are included data on beneficiaries of EU funds from 2007 onwards, as well as data on beneficiaries of the European Development Fund (EDF) from 2010 onwards. The financial information contained in the FTS is expressed in commitments and not in payments - it corresponds to the amounts awarded (“committed”) to beneficiaries in a given year. The data is updated at least in June the year following the financial year concerned. For example, data on the year 2014 will be published in June 2015.
+
+**Centralised Indirect Management**
+
+****
+
+When the management of the fund is delegated to an executive, decentralised or national agency (indirect management), it is the responsibility of this authority to publish beneficiaries data on its website. In case the information is not published directly, the website of the authority should at least contain a link to the website where the information can be found.
+
+The same rules apply to the publication of data on beneficiaries under centralised direct and indirect management.
+
+****
+
+**Shared Management**
+
+For information on beneficiaries under shared management, it the responsibility of Member States to make the data publicly available.
+
+For the 5 ESIF and according to [the Common Provisions regulation](http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32013R1303), Member States must publish data on the name of the beneficiaries, the activity and the amount of public funding allocated. For these funds and the European Agricultural Guarantee Fund, the European Commission provides on its own website links to the websites of Member States with information of beneficiaries under shared management. This concerns beneficiaries of:
+
+-   [the Common Agricultural Policy](http://ec.europa.eu/agriculture/cap-funding/beneficiaries/shared/index_en.htm);
+-   [the European Regional Development Fund and the Cohesion Fund](http://ec.europa.eu/regional_policy/index.cfm/en/atlas/beneficiaries/?lan=en);
+-   [the European Social Fund](http://ec.europa.eu/esf/main.jsp?catId=46&langId=en&list=0);
+-   [the European Maritime and Fisheries Fund](http://ec.europa.eu/fisheries/contracts_and_funding/the_european_transparency_initiative/index_en.htm). 
+
+**Development and Humanitarian Aid to Non-EU Countries**
+
+****
+
+Data is published directly by the European Commission on:
+
+-   Agreements and contracts signed in the field of humanitarian aid during the year concerned, whether financed under the EU budget or the European Development Fund (for ACP countries only): [http://ec.europa.eu/echo/node/2190](http://ec.europa.eu/echo/node/2190);
+-   The beneficiaries of grants and contracts awarded by EuropeAid in the field of development and cooperation: [http://ec.europa.eu/europeaid/work/funding/beneficiaries](http://ec.europa.eu/europeaid/work/funding/beneficiaries). 
+
+* * * * *
+
+- [Return to Beginning](../)
+- [Prev: Where Does the Money Go?](../funds/)
+- [Next: Journalistic Investigations and Transparency Projects to Follow the Money](../research/)
+
+{% include_relative footnotes.md %}

examples/openspending/content/resources/gift/acknowledgements.md

@@ -0,0 +1,44 @@
+---
+title: Acknowledgements
+---
+
+## Acknowledgements
+
+This report was commissioned by the [Global Initiative for Fiscal Transparency](http://fiscaltransparency.net/)
+
+The [Global Initiative for Fiscal Transparency (GIFT)](http://fiscaltransparency.net/) is a multi-stakeholder action network working to advance and institutionalize global norms and significant, continuous improvements on fiscal transparency, participation, and accountability in countries around the world.
+
+Authors: 
+
+* Lucy Chambers, <lucy.chambers@okfn.org>
+* Velichka Dimitrova, <velichka.dimitrova@okfn.org>
+* Rufus Pollock, <rufus.pollock@okfn.org>
+
+For more information contact: <gift-report@okfn.org>
+
+Special thanks to:
+
+* Martin Tisne
+* Randall Kemp
+* Libby Haight
+* Sanjeev Khagram
+* Friedrich Lindenberg
+* Vivek Ramkumar
+* Laura Newman
+* **the many contributors who provided sections for the report.**
+
+We would also like to thank the Noun Project artists:
+Andrew Cameron, Denis Frezzato, John Caserta, Marwa Boukarim, Øystein W. Arbo, Pedro Lalli, Scott Lewis, SZ
+
+This is the first edition of the Technology for Transparent and Accountable Public Finance Report
+
+*First Published May 2012*
+
+This is a report by: 
+
+<pre>
+Open Knowledge Foundation
+37 Panton Street, Cambridge
+CB2 1HL
+United Kingdom
+</pre>

examples/openspending/content/resources/gift/bibliography.md

@@ -0,0 +1,46 @@
+---
+title: Bibliography
+---
+
+## Further Resources
+
+* *Avila, R. et al., 2010*. **Global mapping of technology for transparency and accountability New technologies**. Technology, 133(7), pp.189-204. Available at: <http://www.transparency-initiative.org/reports/global-mapping-of-technology-for-transparency-and-accountability>.
+
+* *Bertot, J.C. et al., 2010*. **Social media technology and government transparency**. Computer, 43(11), pp.53-59.
+
+* *Bilbao-Osorio, B. & Dutta, S., 2012*. **The Global Information Technology Report 2012: Living in a Hyperconnected World**, Available at: <http://reports.weforum.org/global-information-technology-2012/>.
+
+* *Centre for Good Governance, 2009*. **Best Practice Notes on Social Accountability Initiatives in South Asia: Social Audit of NREGS (AP) in Andhra Pradesh**, Available at <http://bit.ly/CCG-AndhraPradesh>.
+
+* *Chipchase, J. & Lee, P., 2011*. **Mobile Money Afghanistan, Institute for Money, Technology and Financial Inclusion**, Available at <http://bit.ly/mobile-money-af>.
+
+* *Devarajan, S. & Reinikka, R., 2003*. **World Development Report 2004: Making Services Work for Poor People**, World Bank and Oxford University Press.
+
+* *Gigler, B.S., Tanner, B. & Kiess, J.., 2011*. **Enhanced Social Accountability through Open Access to Data: Geomapping World Bank Projects**. Development Outreach.
+
+* *International Budget Partnership*, **Guide to Transparency In Public Finances**, Available at: <http://internationalbudget.org/wp-content/uploads/Looking-Beyond-the-Budget.pdf>
+
+* *International Budget Partnership, 2010*. **Open Budgets Transform Lives: Open Budget Survey**, Available at: <http://internationalbudget.org/wp-content/uploads/2011/06/2010_Full_Report-English.pdf>.
+
+* *Kuriyan, Renee; Bailur, Savita; Gigler Bjorn-Soren and Park, K.R., 2011*. **Technologies for Transparency and Accountability-Implications for ICT Policy and Implementation**, (DRAFT) Open Development Technology Alliance, The World Bank, Available at: <http://scr.bi/wb-report-draft>.
+
+* *Lennox, D.*, **M-PESA Agent Anti Money Laundering Procedure**, pp.1-9, Available at <http://bit.ly/m-pesa-money-laundering>
+
+* *Moon, S. & Mills, Z.*, **Practical approaches to the aid effectiveness agenda, Overseas Development Institute, Working Paper 317**, Available at: <http://www.publishwhatyoufund.org/files/Practical-Approaches-to-the-Aid-Effectiveness-Agenda.pdf>
+
+* *Nino, E., 2010*. **Access to Public Information and Citizen Participation in Supreme Audit Institutions (SAI), World Bank Institute, Governance Working Paper Series**, Available at: <http://bit.ly/CitizenParticipationSAI.>
+
+* *Open Knowledge Foundation*, **Open Data Handbook**. Available at: <http://opendatahandbook.org/>.
+
+* *Participatory Budgeting Unit*, **The role of new technology in Participatory Budgeting**, Available at:  <http://www.participatorybudgeting.org.uk/documents/Discussion%20paper%20FINAL%20version.pdf>.
+
+* *Ribeiro, M.M., Vaz, J.C. & Matheus, R.*, **Transparency in the Portals of Brazilian Federal Government: The cases of E-procurement Portal and Transparency Portal**. In First Global Conference on Transparency Research, Rutgers University, Newark. Available at: <http://bit.ly/brazilian-portals>.
+
+* *Ramkumar, V., 2008*. **Our Money, Our Responsibility: A Citizens’ Guide to Monitoring Government Expenditures**, International Budget Project (IBP).
+
+* *Solana, M., 2004*. **Transparency Portals: Delivering public financial information to Citizens in Latin America. THINKING OUT LOUD V Innovative Case Studies on Participatory Instruments**, The World Bank, Available at <http://bit.ly/ThinkingOutLoud>
+
+* *The Transparency and Accountability Initiative, 2011*. **Opening government: A guide to best practice in transparency, accountability and civic engagement across the public sector**, Available at: <http://www.transparency-initiative.org/wp-content/uploads/2011/07/Opening-Government2.pdf>.
+
+* *Wohlschlegel, K. & Baxandall, P., 2012*. **Following the Money**, Available at: <http://www.uspirg.org/reports/usp/following-money-2012>
+

examples/openspending/content/resources/gift/chapter1-2.md

@@ -0,0 +1,17 @@
+---
+title: Context
+---
+
+## Context
+
+We are living in a technical revolution. The Internet allows almost instant sharing of information, computing power permits complex calculations, and visualisations and tools to allow better analysis and comprehension of large datasets. Technology facilitates two-directional information flow, not only from governments but also to governments, allowing them to collect meaningful and structured feedback on fiscal policies. Instead of commissioning external consultancy firms to estimate citizens’ opinions and needs, governments could use the direct input from citizens through the technological and communication tools.
+
+Over recent years, the world has become increasingly [“hyperconnected”](http://reports.weforum.org/global-information-technology-2012/), driven by the rise of big data, mobile technology and social media. Access to the Internet has transformed the relationships between individuals, consumers and enterprises, civil society, citizens and the state, enabling accessible and immediate services and direct citizen participation. The possibility to be interconnected and to communicate instantly has created opportunities for informing citizens and including them in the decision-making processes of governments.  Moreover, the price of technology is dropping, sparking extensive access to mobile technology, even in developing countries, which brings exciting opportunities for outreach, education and feedback.
+
+Well-established, low-cost technologies such as the radio, SMS  and print hold strong, and continue to be a stronghold for outreach and communication of messages. The open data movement, particularly opening up government information through data-portals, would most likely not have been financially feasible even a few decades ago.
+
+These developments hold promising new possibilities for fiscal transparency and accountability, more information, better information, new possibilities for reaching out to people who would not normally interact with government, new possibilities to collect feedback and new possibilities to present information for analysis. In this report we examine two main areas: technology for transparency and technology for accountability around public finance. We will look in particular at who builds these tools, who uses them, and who benefits from them.
+
+With initiatives such as the [Open Data Strategy for Europe](http://bit.ly/EUopendata) soon due to come into force, and similar emphasis on opening up government data in other parts of the world, it is prudent for governments to start to look into new ways to open up their data, to save time, money and effort in opening up a backlog of data and to get necessary workflows in place.
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter1-3">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter1-3.md

@@ -0,0 +1,219 @@
+---
+title: Highlights, Gaps & Recommendations
+---
+
+## Highlights, Gaps and Recommendations
+
+
+> “There’s a really interesting dynamic interaction between work on data availability and quality, and on building tools, visualisations etc. that work with it. Without good data it’s hard to use it for transparency and accountability, and to build good tools. But without tools to start using the data, it’s hard to discover where it needs to improve.”
+
+> <small>Tim Davies - Practical Participation</small>
+
+In this section, we summarise the key highlights from the various sections of the report. The highlights are arranged by section of the report and followed by a few notes on the ‘gaps’: areas which, in our opinion, merit further investigation or have thus far lacked resources or coordination to make happen. Each section concludes with ‘recommendations’: We have tried to stagger these suggestions, ranging from first steps, easily obtainable goals which are likely to be feasible in the short term, to ideal scenarios, desirable end goals, which will take more commitment and longer term planning.
+
+### Data Portals
+
+In [this section](../chapter2-intro) we examine four government data portals from across the globe. The aim is to highlight government perspectives of making fiscal data available to the public. We focus on motivations for creating the service in the first instance, what features and data are offered, successes and outcomes, challenges and areas for improvement and what resources are needed to assemble and maintain such a site.
+
+#### Highlights
+
+* Data portals help address a fundamental requirement to make data accessible and discoverable once it has been made available: If data is spread across hundreds of websites and is difficult for people or machines to find, much of its value will be lost.
+* The reason many government catalogues are link sites rather than data stores is often political, not just technical: Departments want to remain in charge of their data. Local departments often have to be encouraged to give up their data and be persuaded of the value of holding data centralised resource.
+* Data portals can be useful both within and outside of government. Lack of information sharing affects civil servants and policy-makers -- not just those outside of government. This was highlighted in the OpenDataBC case study, where approximately one third of traffic to the citizen-facing data portal came from IP addresses within government.
+* Keep it simple: A portal can be very simple initially and developed in response to user and government demands. Moreover, today exist off-the-shelf open-source solutions: Governments are increasingly learning the benefits of open-source solutions, (flexibility, adaptability, auditability etc.) but further work needs to be done to [encourage more widespread adoption](http://open-source.gbdirect.co.uk/migration/benefit.html
+).
+* An explicit open data policy is not needed to start an open data portal: One can begin with datasets that are already open or which are “voluntarily” donated by departments. In the UK’s case the portal preceded, and helped catalyse, development of a formal open data policy.
+* Data portals could become of the most important technological tools of the government with regard to public spending monitoring and control.
+
+
+#### Gaps
+
+* Greater promotion of open-source tools for open data
+* More explicit opportunities for citizen participation. Some projects highlight good examples of how participation can fit within existing government workflows. See participation section for more, largely civil-society driven projects, which may provide inspiration - or source code - for some government-led initiatives promoting direct participation.
+
+#### Recommendations
+
+<div class="well homework">
+    <strong>GIFT Actions: Promoting Open Data as the Raw Fuel for Technology</strong>
+    <br></br>
+    <ul>
+        <li>Members of GIFT should themselves ensure - whether they are national governments or international organisations - that they publish their fiscal and other financial information in full and in machine-readable formats.</li>
+        <li>Promote publication of key budget information as machine-readable data. Work with organisations such as the International Budget Partnership to include a related criterion in the Open Budget Survey. Set up online webinars / workshops showing how data is used after its publication.</li> 
+        <li>GIFT should initiate the creation of a light-weight, demand-driven standard for the release of structured expenditure information to enable its comparability between countries.</li>
+        <li>Work closely with the Open Government Partnership to promote release of datasets which are key to contextualising financial data, such as economic indicators or procurement data.</li>
+        <li>Promote the release of sub-national data, perhaps via local partners, as well as national level.</li>
+    </ul>
+</div>
+
+**First Steps**
+
+* Fiscal data must be openly licensed and available in machine readable format in bulk. Be strict about licensing: make sure that all datasets on the portal are released under a proper [open license](http://OpenDefinition.org/).
+* Engage with your user community both within and outside government. What data do they want, in what form? How can they report issues effectively and easily? Have explicit and prominent contact links so that citizens can report issues. Even better, allow them to browse past issues or see the clarification which others have received in answer to other questions e.g. through online discussion forums.
+* Engage with your provider community: civil servants and others working inside government who will be publishing data using the site. What can be done to make this process as easy and rewarding as possible?
+* Be timely: particularly for financial information, having current data is key to enabling re-use with an impact.
+* Open tools for open data: use free and open-source tools wherever possible.
+
+**Next Steps**
+
+* Monitor data release, quality and usage and reporting this to relevant decision-makers becomes increasingly important as the project matures.
+
+
+### Finding and Using Data
+
+In [this section](../chapter3-intro) we asked the open spending data community - who come from a variety of backgrounds; research, technical, media etc., what fiscal data they require, what they want to do with it, and importantly, how easy it is to obtain and use. Through a series of structured surveys, we established the following:
+
+#### Highlights
+
+* Fiscal transparency is about more than budgets: Users will often require more than just basic budget information to hold governments to account. Fiscal transparency should therefore not be limited to making these key budget documents accessible. Company and procurement data are common requests, as are audit reports and sub-national budget information.
+* We received responses to the questionnaire from eighteen countries, spread around the globe. Six respondents mentioned that they used a government transparency portal to get their data, three used FOI requests, seven collected the data themselves from various sources and two said they had not been able to acquire the data.
+* ‘The current status of open data’: Feedback was mixed. Some respondents praised the ease with which they acquired the data, but in the majority of cases, it was not easy to use the data that they had obtained. Barriers included:
+* Machine readability: The most common complaint was non machine-readability of data - usually the complaint was about PDFs.
+* Poor quality: Frequently there are problems with the data such as lack of care for the process of creating the data, poor structure, too aggregated and inconsistencies in structures between years.
+* Disappearing data or hidden data: Several users mentioned that historic data disappeared from government websites and two could not get access to the data because of paywalls. A couple of more technical users of the data said an API for the data would be useful.
+* There is often a lack of supporting documentation to complement data releases, meaning that users cannot find out what is actually contained within a dataset.
+
+#### Gaps
+
+* Machine-readable data
+* Data-management systems which ensure good-quality, consistent data
+* Preservation of historical data
+
+#### Recommendations
+
+<div class="well homework">
+    <strong>GIFT Actions: Technology Outreach and Idea Sharing</strong>
+    <br></br>
+    <ul><li>Promote experience sharing between governments through workshops and forums in existing social media platforms. Governments need to share the lessons learned in using technology to promote transparency, accountability and participation across different countries.</li>
+        <li>Involve civil society actors who use data and services provided by governments, to review and share ideas about existing and planned initiatives. Such actors could form advisory-panels on particular topics e.g. company data.</li>
+        <li>Create a well-maintained index of existing initiatives both an the demand and supply side, including methods of data release, re-use and presentation, discussion or activism enabled by such information.</li>
+        </ul>
+</div>
+
+**First Steps**
+
+* Make data available in a machine-readable, fully disaggregated form
+* Provide some adequate basic information about the released data
+
+**Next steps**
+
+* Provide of machine-readable forms of important datasets such as companies registries, contract awards and underlying tender specifications
+* Provide structured, machine-readable information on budgeting process, e.g. important dates (publication dates, points in the process where feedback is possible), amendments and information on “who does what” in budgeting issues
+* Provide comprehensive documentation to accompany the documents i.e. read-me files explaining what a given dataset, data-dictionaries, glossaries
+* Capacity-building, connecting policy and data processing/analysis expertise
+* Promote data sharing of intermediate results e.g. integrated datasets produced by researchers, journalists, advocacy organisations and developers and the creation of data commons for processed output of re-use applications
+
+**Ideal scenario**
+
+* Open-source tools used within government for production and consumption of data
+* Long-term archives of data on “neutral ground” i.e. not run by states nor advocacy groups, something like the Internet Archive for data
+* Integration of budget, procurement, companies and contracts, metrics/evaluation and spending information.
+* Real-time access to data via an API: Note that this is not relevant in all cases, e.g. for budget information that only changes once per year.
+
+
+### Standards for Fiscal Data
+
+[This section](../chapter4-intro) examined whether having standards for fiscal data could provide a solution to some of the usability issues described by the users of the data in the previous section, by looking at two internationally recognised standards, IATI and XBRL.
+
+#### Highlights
+
+* Standards enable a distributed rather than a centralised approach in data publication and use. No or very limited general fiscal data standards are currently in existence and in use.
+* XBRL is establishing itself as a worldwide standard for accounting information and may grow into a standard for general ledger data. It is however complex and focused on the traditional accounting domain - especially around balance sheets - which may limit its relevance and benefits as a format for transparency purposes.
+
+#### Gaps
+
+* A standard for transaction level spending data.
+* Lack of widely-adopted standards for entity identifiers, which would greatly facilitate merging and matching
+* Tools to merge and annotate the data
+* Training and quality assurance (validation) tools
+
+
+#### Recommendations
+
+<div class="well homework">
+    <strong>GIFT Actions: Technology Review</strong>
+    <br></br>
+    <ul><li>Review existing projects and and solutions from the point of view of both governments and civil society actors. <a href="http://civiccommons.org/apps">Civic Commons Marketplace</a> as well as the <a href="http://bit.ly/TTAPF-projects">appendix</a>, but a larger-scale, community effort will be required to keep this up to date.</li>
+        <li>Based on feedback from the above refine and maintain an online, collaborative catalogue of technical and policy options (perhaps as a wiki) which can guide governments about existing solutions and experiences. 
+        </li></ul>
+</div>
+
+**First Steps**
+
+* Use well-known, commonly used formats such as CSV, XLS or XML for the release of data.
+* Adopt existing coding conventions for shared entities, such as countries e.g. ISO-3166 and EU NUTS codes. Publish information on the identifiers used for companies and make sure they are not entangled in IP e.g. DUNS numbers or behind a paywall.
+* Publish additional information on the coding schemes used, such as functional or economic classifications, charts of account.
+
+**Next Steps**
+
+* Enter standardisation process, beginning with core taxonomies. Additional work on aligning ‘spines’ of data e.g. mapping IATI data onto COFOG compliant budgets.
+* Geocode data using re-usable, openly licensed coding services to avoid inclusion of proprietary IP.
+* Begin using globally unique URIs for companies and other organisations, government bodies, and projects.
+* Critically evaluate the options for re-using IATI or XBRL formats rather than creating new standards specifically for transaction level spending data. Explore options for format convertibility.
+
+**Ideal Scenario**
+
+* A global spending data registry which lists spending data from around the world in a standard format - similar to the IATI registry for aid information.
+
+
+### Where Does the Money Come From? Where Does the Money Go? and The Invisible Money
+
+#### Highlights: Tools for Data
+
+* Civil society is often cited as the target users for transparency portals, as was the case for the Sierra Leone Mining Repository. However, in reality, 65 % of the users of this platform are investors, donors and other governments, who have knowledge of complex legal terminology and sector-specifics. The level of technical knowledge required to understand this data means that similar audiences may be the main user groups for other such tools.
+* Design of tools for end-users are most effective when they bear in mind less-web-savvy users or users who will have to work in lower bandwidth or with older computers. Options such as saving and printing user-generated reports could be very useful for these purposes.
+* “Data is not always actionable simply because it is available.” Having the possibility to download the data does not guarantee that one would be able to use it or understand its structure and source.
+* Open source tools allow rapid dissemination of approaches and the adaptation of tools to different contexts internationally. For example, the visualizations from OpenSpending have been widely and independently re-used (e.g. to power the Hungarian site [“A mi Penzünk”](http://amipenzunk.hu/) and the Czech site [BudováníStátu.cz](http://budovanistatu.cz/)).
+* Information to hold individual politicians personally accountable for their financial choices is highly desirable, particularly for journalists, and could have a great impact, particularly in situations where individual politicians wield choice over discretionary funds.
+
+#### Highlights: Participation
+
+* Technology can add additional channels for participation to reach out to people who might otherwise be hard to reach, or make it easier for those who already do participate. Yet technology is not a substitute for face-to-face contact.
+* Technology is not a fix-all. If a tool is to be successful, it will most likely target a specific aspect of the Participatory Budgeting-cycle. A strategy can target a particular part of the PB process rather than being a super-app that will work at every stage.
+* Conversations should be tightly facilitated in the idea collection and deliberation phases to ensure that the conversation remains structured and duplicate or irrelevant ideas do not distort its clarity. Governments should also ensure that they have the response capacity to be able to deal with the potentially large amounts of feedback, which could ensue before implementing a technical solution to idea collection.
+
+#### Highlights: Social Auditing
+
+* Social audits using technology offer a possibility of close to real-time feedback and reporting.
+* Technology, e.g. mobile, can help transform traditional labour-intensive social auditing practices, which gives a possibility to scale-up corruption monitoring.
+* SMS technology has the possibility of reaching out to rural areas with limited Internet access.
+* Many of the projects featured in this section rely on sub-national level data to function: It is at local and regional level that many of the services from which citizens benefit most tangibly  are delivered e.g. infrastructure, healthcare, schooling.
+* Another type of data, which could be key for stimulating citizen feedback on an individual level, is data on personal entitlements, which might motivate citizens to provide feedback on whether they received their personal allowance.
+
+#### Gaps
+
+* Raw, machine-readable data is vital.  Having to extract structured data from unstructured documents (e.g. PDFs and Word documents) prior to use creates work, could introduce errors (e.g. from manual transcription) and can lead to supply-side data-quality issues, e.g. if the government faces bad data management practices.
+* Very few projects are currently able to trace the difference between planned and actual expenditure. Sunlight’s work in the United States is one of the few examples of success in this area.
+* Good, open-source tools for doing entity matching and extraction
+* Sub-national level spending data
+* Data on citizens’ personal entitlements
+
+#### Recommendations
+
+<div class="well homework">
+    <strong>GIFT Actions: Technology Sharing</strong>
+    <br></br>
+    <ul>
+        <li>Promote the use of promising projects from the Technology Review stage. Liaise with funding bodies to ensure open-source solutions are available for all. </li>
+    </ul>
+</div>  
+        
+**First Steps**
+
+* Build public-facing tools, targeted at enabling expert oversight, e.g. by investors and other governments, who have both incentives to scrutinise this data and pre-existing knowledge to enable them to do so.
+* Promote use of these tools  to infomediaries, who perform the role of translating complex information for the benefit of the public
+* Related datasets: Fiscal data is most useful when related core datasets (e.g. reference identifiers, geospatial information) are  also available
+* More and better data: Access to good quality sub-national transaction-level data on expenditure; access to data on personal entitlements
+
+**Next Steps**
+
+* Contextualise the data in a way that is suitable for a layman audience - how was it generated? What is and is not covered?
+* Build tools using the data that has been released as part of the ‘first steps’, in order to allow greater understanding of information for a layman audience. For example, citable, pre-computed statistics (as long as the methodology behind generating them is clear), which have been generated by experts are extremely attractive for those looking to use them in reports and for the press.
+* Closer collaboration between advocacy NGOs and journalists.
+
+**Ideal Scenario**
+
+* Finishing off the great work done so far in this area in compiling country-level reports into one place and building a sortable online database.
+* Extensive capacity-building for journalists to enable them to analyze and present data.
+* Social Audit Portals similar to Andhra Pradesh with near real-time access to financial data.
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter1-4">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter1-4.md

@@ -0,0 +1,106 @@
+---
+title: Key to Case Study Icons
+---
+
+The icons below are used as a quick overview for the projects listed in this report. You will find them arranged in a row at the top of each case study.
+
+
+## Fiscal Scope
+
+<table cellpadding="3">
+<tr>
+
+<td><img alt="Spending Side" src="http://farm8.staticflickr.com/7088/7285692280_90b76e026b_t.jpg"
+	width="50" height="50"/>
+</td>
+<td>
+<img alt="Spending Side" src="http://farm8.staticflickr.com/7097/7285692328_347f4d560c_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Off-Budget" src="http://farm9.staticflickr.com/8150/7285691984_5a3c52e5b2_t.jpg" width="50" height="50"/>
+</td>
+
+</tr>
+<tr>
+<td>Revenue Side</td>
+<td>Spending Side</td>
+<td>Off-Budget</td>
+</tr>
+</table>
+
+
+## Aims of the Project
+
+<table cellpadding="3">
+<tr>
+
+<td><img alt="Publish Better Data" src="http://farm8.staticflickr.com/7211/7285692458_bb8a771171_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Educate Citizens" src="http://farm8.staticflickr.com/7226/7285691742_9934b69422_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Facilitate Direct Participation" src="http://farm8.staticflickr.com/7083/7285691814_3985558f75_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Get Feedback to Policy Makers" src="http://farm8.staticflickr.com/7078/7285691912_7b78aed3dc_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Analyse and Understand Data" src="http://farm8.staticflickr.com/7234/7285691862_8867e2dfb7_t.jpg" width="50" height="50"/>
+</td>
+
+
+
+</tr>
+<tr>
+<td>Publish Better Data</td>
+<td>Educate Citizens</td>
+<td>Facilitate Direct Participation</td>
+<td>Get Feedback to Policy Makers</td>
+<td>Analyse and Understand Data</td>
+</tr>
+</table>
+
+
+## Technology
+
+<table cellpadding="3">
+<tr>
+
+<td><img alt="Mobile Technology" src="http://farm8.staticflickr.com/7104/7285692036_e162601e31_t.jpg" width="25" height="25"/>
+</td>
+<td>
+<img alt="Web-based Technology" src="http://farm8.staticflickr.com/7225/7285691644_3bdc21a090_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Offline and Print on Demand" src="http://farm8.staticflickr.com/7083/7285691814_3985558f75_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Data Visualisation and Maps" src="http://farm8.staticflickr.com/7237/7285691690_9deb732ff0_t.jpg" width="50" height="50"/>
+</td>
+<td>
+<img alt="Formats and Standards" src="http://farm8.staticflickr.com/7083/7285692382_03cbe2d4a3_t.jpg" width="50" height="50"/>
+</td>
+</td>
+<td>
+<img alt="Social Media" src="http://farm8.staticflickr.com/7237/7285692088_5466b1461e_t.jpg" width="50" height="50"/>
+</td>
+</td>
+<td>
+<img alt="Radio" src="http://farm8.staticflickr.com/7237/7285692216_756aa84a83_t.jpg" width="50" height="50"/>
+</td>
+
+
+</tr>
+<tr>
+<td>Mobile Technology</td>
+<td>Web-based Technology</td>
+<td>Offline and Print on Demand</td>
+<td>Data Visualisation and Maps</td>
+<td>Formats and Standards</td>
+<td>Social Media</td>
+<td>Radio</td>
+</tr>
+</table>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter2-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter1.md

@@ -0,0 +1,47 @@
+---
+title: Introductions
+---
+
+## Chapter 1: Introductions and Methodology
+
+### Introduction
+
+This report, “Technology for Transparent and Accountable Public Finance”, was commissioned by the Global Initiative for Fiscal Transparency (GIFT) in February 2012 in order to assist the GIFT in assessing the potential of technology to aid transparency and accountability in relation to governments’ fiscal activities. 
+
+This reports provides examples of projects around the world that are using technology (web, mobile or otherwise) to further aims of fiscal transparency. We focused on projects which: 
+
+* Publish more or better data related to fiscal processes (aid, revenues, budgets, audits, etc. — see below), 
+* Help understand this data through the creation of better visualisation and data analysis tools, 
+* Educate citizens about fiscal processes, and assist civil society organisations in promoting accountable governance, 
+* Facilitate direct participation in fiscal matters through participatory budgeting, citizen auditing, etc.,
+* Provide policymakers with complete and reliable data relevant to their work, enabling them to make better decisions. 
+
+We have sought to find projects which reflect all stages of the fiscal process: 
+
+* Looking at [where the money comes from](../chapter5-intro): both getting more data released and building analysis and visualisation tools in the context of revenue processes e.g. taxation, extractive industries, etc., [c]
+* Monitoring [where the money goes](../chapter6-intro): presenting data about the budgeting process and getting citizens involved in fiscal processes e.g. through participatory budgeting and comparisons of planned and retrospective budgets,
+* [The invisible money](../chapter7-intro): improving public understanding of state-owned or semi-state owned enterprises, off-budget information and social audit projects which verify whether official money is being concealed or is not being spent according to official plans – information which often is not published as part of current budgeting practices.
+
+In each case, we considered questions such as:
+
+* Who are the users and audiences of the project?
+* What are their motivations and what skills are required?
+* What are the successes, failures and limitations?
+
+For each project, we have highlighted strengths and weaknesses based both on our own experience of developing tools, and also by seeking feedback from civil society organisations and the technical community. This report also aims to highlight the gaps: In some instances, cutting-edge technology is being used in fields besides public finance, which may merit further exploration; in other cases, we highlight points in the budgeting cycle which are currently underserved by technical tools.
+
+### Methodology:
+
+We have selected case studies that are appropriate across:
+
+* Different levels of literacy and access to technology
+* Different budgets
+* Government-led, civil society led and citizen led initiatives
+
+The research was a combination of qualitative interviews and direct inputs from case study representatives through online questionnaires. 
+
+### Overview of Projects
+
+![Project Overview](http://farm8.staticflickr.com/7237/7273947644_4799a7e720_o.png)
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter1-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter10-intro.md

@@ -0,0 +1,12 @@
+---
+title: Appendix
+---
+
+
+## Final Observations and Review
+
+The appendix to this report is a consolidated list of projects extracted from the case studies. In addition to this are  other projects, which there was not space to include in the main body of the report, that are however useful examples of tools, approaches and contacts for those interested in fiscal transparency. The list includes the name of the project and the organisation, the geographical focus - where it was implemented as well as the local and national scope, the stage in the fiscal process and the type of technology, link to the tool and a short background. You can access the list [here](http://bit.ly/TTAPF-projects).
+
+We would like to encourage users to help keep this list up to date - if you would like to add a project, please [email us](mailto:gift-report@okfn.org).
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../bibliography/">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter2-1.md

@@ -0,0 +1,138 @@
+---
+title: CKAN in Data.Gov.Uk
+---
+
+## Out of the Box Solution: CKAN in Data.Gov.UK
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="no" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="no" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Country</th>
+        <td colspan="2">United Kingdom</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://data.gov.uk/">data.gov.uk</a></td>
+    </tr>
+</table>
+
+<img alt="OpenSpending Data.Gov.Uk" src="http://farm8.staticflickr.com/7093/7274090358_35c6eff43d_o.png" class="screenshot" />
+
+### Background
+Data.Gov.UK is the UK Government’s official open data portal. The site provides a central “way into the wealth of government data” and aims to make that data “easy to find; easy to license; and easy to re-use.” Data.Gov.UK was launched in closed Beta at the start of October 2009 and entered public beta in January 2010. The project is ongoing at the present time.
+
+Data.Gov.UK has been built on a combination of CKAN, the open-source data portal software developed by the Open Knowledge Foundation, and Drupal, the open-source CMS.
+
+Initial requirements for Data.Gov.UK were data catalog capabilities (entering, editing, listing, and searching datasets) combined with basic CMS features (site content, blog, theming etc). The use of open-source plus the use of existing components which allowed for rapid development were desired (the initial prototype was developed in less than a month). Over time a variety of new requirements have arisen, most notably some need for data storage and presentation.
+
+Outcomes have been very positive. The UK government is continuing to use and develop Data.Gov.UK and the site has a global reputation as a leading exemplar of a government data portal. The system has successfully handled growth from a few dozen datasets to many thousands of datasets and a concomitant growth in site traffic, and the site has played a significant enabling role in the UK government’s development of its transparency and open data agenda.
+
+### Available Fiscal Data
+1. In 2010 the UK government committed to the ongoing release of a substantial amount of open fiscal data. Specifically, in the Prime Minister’s letter of 31 May 2010 the [Government committed to](http://www.number10.gov.uk/news/letter-to-government-departments-on-opening-up-data/):
+2. Historic COINS spending data to be published online in June 2010.
+3. All new central government ICT contracts to be published online from July 2010.
+4. All new central government lender documents for contracts over £10,000 to be published on a single website from September 2010, with this information to be made available to the public free of charge.
+5. New items of central government spending over £25,000 to be published online from November 2010.
+6. All new central government contracts to be published in full from January 2011.
+7. Full information on all DFID international development projects over £500 to be published online from January 2011, including financial information and project documentation.
+8. New items of local government spending over £500 to be published on a council-by-council basis from January 2011.
+9. New local government contracts and tender documents for expenditure over £500 to be published in full from January 2011.
+10. Other key government datasets:
+* Names, grades, job titles and annual pay rates for most Senior Civil Servants with salaries above £150,000 to be published in June 2010.
+* Names, grades, job titles and annual pay rates for most Senior Civil Servants and NDPB officials with salaries higher than the lowest permissible in Pay Band 1 of the Senior Civil Service pay scale to be published from September 2010.
+
+The actual delivery of individual commitments obviously took some time, but, for example:
+
+1. The COINS database was released in June 2010. The COINS database is the central government database for budgetary information, used by HM Treasury to manage budgeting and outturn against budget from all departments. This data was published on <http://data.gov.uk/> at <http://data.gov.uk/dataset/coins> (More information here: <http://thedatahub.org/dataset/coins-data>).
+2. In November 2010 the government released, and committed to ongoing monthly release of, detailed departmental and local authority transactional spending data (all spending above £25,000 for departments and above £500 for local authorities). Departmental spending data was published onto <http://data.gov.uk/> while local authorities usually published their data onto their local website or data catalog.
+3. In Autumn 2010 details of central government contracts became available online.
+
+In July 2011 the Prime Minister issued another [letter](http://www.number10.gov.uk/news/letter-to-cabinet-ministers-on-transparency-and-open-data/ ). In addition to reviewing performance against commitments from the previous year, the letter proposed various improvements and extensions in relation to fiscal data (note that almost all major fiscal information was now open so there was little to do in terms of new data release):
+
+1. All government spending data to include plain English descriptions explaining the scope and purpose of every transaction, from September 2011
+2. Every department, working with the Cabinet Office transparency team, to produce an action plan in November 2011 for improving the quality and comparability of data
+3. Unique reference indicators to be introduced by DBIS and HMRC beginning in December 2011. These will enable the public to track more easily the interaction between companies and government bodies
+4. Working with the purchase and payment card providers to provide a consistent method of reporting government procurement card spend data for transactions above £500 in value, so this is available for publication on departmental websites, from end September 2011.
+
+
+### Overview of Features
+Data.Gov.UK acts both as a data portal and as a home for some of the government’s information on transparency and open data (for example, minutes and notes from the Transparency Board). As a data portal, its main features are:
+
+1. Publish and find datasets: full data catalog with rich search capabilities
+2. Store and manage data: the majority of the datasets that data. gov.uk lists are hosted elsewhere e.g. on individual departments’ websites. However, there has been the need to store and manage data and an upcoming release will see these features substantially enhanced.
+3. Community and social features such as the ability for users to list applications or ideas that relate to a dataset, comment, share dataset information on social media, and subscribe to RSS/Atom feeds to be kept up to date with the latest developments.
+4. Federation and Harvesting: data.gov.uk acts as the UK’s hub for geospatial metadata aggregation in relation to the EU’s INSPIRE directive and therefore harvests information on geospatial datasets from a large number of other data catalogs and hubs.
+5. Geospatial: Add and manage geospatial information about a dataset, view this information on maps and incorporate into search queries.
+6. Rich API: Access to all dataset information over an API (Application Programming Interface).
+
+While this is a very rich feature-set it should be emphasised that essentials of a successful data portal can be substantially less -- data.gov.uk itself in its original incarnation had many fewer features. A data portal in its simplest form need only have a mechanism for easily listing datasets (both in human-readable and machine-readable form) -- and datasets may “point out” to data stored on other sites (e.g. individual ministries’ or departments’ websites) rather than being stored on the portal itself (though over time, there may be a need to store data, at least for archival purposes).
+
+### Successes
+Data.Gov.UK has won widespread recognition as an exemplar data portal and its influence has been felt widely within the international community. In the UK, it has become the online home of the UK Government’s open data and transparency efforts and the data published on the site has been widely used and reused by companies, journalists, CSOs, and citizens.
+
+By providing a clear, and very visible, home for UK government open data it has also played a direct role in driving forward the open data and transparency agenda -- departments were already releasing datasets onto the site voluntarily *before* there was any specific policy mandating this, and publicity and interest around the site at its early stages from developers, media and others helped to galvanize further policy developments.
+
+In addition, data.gov.uk has played a significant role in the development of a very clear open licensing policy for UK government data by ensuring that all datasets found on the site are under an open license -- the [Open Government License](http://www.nationalarchives.gov.uk/doc/open-government-licence/open-government-licence.htm) drafted by the UK Government’s Office of Public Sector Information (now within the National Archives).
+
+
+### Challenges
+Data.Gov.UK has seen some failures. Largely, these relate to processes around data release that are not under the direct control of that project itself (though the project could take steps to ameliorate these problems).
+
+
+To take one example related to fiscal data and the publication of the £25k spending by departments: Because each department publishes individually, this data on government spending is spread across approximately 1000 datasets on data.gov.uk (it can actually be hard to find them all because there are so many and there is no straightforward method to search for them). In addition, not all data is published in the correct format and some data links disappear as departments move data on their website. While ultimately this is a process issue, data.gov.uk have been taking steps to help improve this: for example, by ensuring consistent tagging of datasets when they are created, automatically checking datasets on a regular basis for broken links, developing validators to ensure that data is provided in a consistent format, and developing reporting tools so that Ministers and managers can get an overview of the process.
+
+
+### Project Resources
+The project was initiated in September 2009 with a go-live data 1-month later for the first version of the site. This target was met and a closed beta started in early October 2009. In January 2010 the site was made public.
+
+<table border="1" padding= "1em">
+    <tr>
+        <td><strong>Phase</strong></td>
+        <td><strong>Duration</strong></td>
+        <td><strong>Full-time human resources</strong></td>
+    </tr>
+    <tr>
+        <td>Planning and data-collection</td>
+        <td>1 month</td>
+        <td>3</td>
+    </tr>
+    <tr>
+        <td>Design and implementation</td>
+        <td>3-4 (person) months (up to closed beta stage), 8 person months (up to launch)</td>
+        <td>3-6</td>
+    </tr>
+    <tr>
+        <td>Ongoing</td>
+        <td>Since Jan 2010</td>
+        <td>3-6</td>
+    </tr>
+</table>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter2-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter2-2.md

@@ -0,0 +1,125 @@
+---
+title: DataBC Open Data Portal of British Columbia
+---
+
+## DataBC Open Data Portal of British Columbia
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="no" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="no" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Country</th>
+        <td colspan="2">Canada</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://www.data.gov.bc.ca/">www.data.gov.bc.ca</a></td>
+    </tr>
+</table>
+
+<img alt="DataBC" src="http://farm8.staticflickr.com/7240/7272465600_0b9d7f60f5_o.png" class="screenshot" />
+
+### Background
+When the Province of British Columbia decided to launch the first Canadian provincial level data portal offering access to more than 2,500 datasets of local and province level data, it followed the need to address both the needs of the community and to transform and civil service system. With a large part of the workforce retiring, the accumulated knowledge and skills had to be transferred and new technology harnessed for realising three main aims:
+
+1. Citizen participation and supporting a community around open data,
+2. Data self-service - transforming the way government is present on the web and providing automated access to information
+3. Business innovation - encouraging the use and re-use of data for fostering innovation through better information and certainly in government policy.
+
+
+### Available Fiscal Data
+revenue side of public finance, pre-budget statements (projected budget), data from quasi-fiscal activities, economic indicators, executive budget proposal, enacted budget, supplementary budget
+
+
+### Overview of Features
+The BC open data portal contains data across a broad spectrum of subjects; access to tools to visualise and analyse the data; and a blog with posts related to the data. The contextualisation of the original document is accomplished through the possibility to “follow” the document, tracing its progression, and the source document link is provided alongside the dataset. The contact form encourages users to ask questions, to enquire about more data when needed, as well as to provide general or specific feedback.
+
+
+DataBC is also trying to balance between servicing the data and supporting the already existing open data community in British Columbia, from whom they received valuable feedback for the project. They are an active Twitter user with a growing number of followers. Organising hackathons also aids developers in using the data and creating data visualisations and apps which present different ideas and communicate messages around data.
+
+
+### Standards, Licenses and Formats
+
+Having used the example of the UK Government’s Open License for Public Sector Information, most of the BC data can be freely reused for commercial purposes, advocacy or personal use.
+
+As the portal provides access to many different types of data, the underlying format and standards vary. There is a need to separate the final format from the file structure - e.g. it could be data in CSV format but the data could be poorly-structured. DataBC had to resolve problems related to the integration of the existing data management systems within the government as no mechanisms for data sharing existed previously.
+
+### Successes and Outcomes
+
+Probably one of the less-expected impacts was seeing how civil servants themselves are using the portal, as about one third of all the traffic originates from government IPs. Technology enables faster access to relevant data within the government departments, contributing to better interconnections between people working e.g. on labour policies and requiring fiscal data on income tax revenues and unemployment benefits. The most popular data turned out to be the financial data of the government, which signals the importance of transparent public finance based on the demand for information. There was also about 20% increase in the number of Freedom of Information (FOI) requests, showing that releasing a small amount of data fuelled wider interest in data.
+
+<div class="well homework">
+<strong> Transparency within government </strong>
+Transparency is not just about allowing civil society access to information. Governments also need information transparency internally and civil servants often suffer from lag times in waiting for data from other departments. Probably one of the less-expected impacts of creating the BC data portal was seeing how civil servants themselves are using the portal: about one third of all the traffic originates from government IPs.
+</div>
+
+### Challenges
+
+Probably the “Holy Grail question” of government data portals is how the data is being used and re-used and what value it generates. As there is no authentication for datasets download and privacy policies only allows the tracking of institutional use, it is difficult to know where the data goes and how it is being used.
+
+It was difficult to switch from a closed system in which data publication was produced in closed formats such as PDFs to one in which the importance of data management, useful metadata etc., is prioritised. Those working on the project experienced an asymmetry in the readiness of various departments to open up their data, for example, those in geospatial data already had much experience with opening up their data, whereas to those in finance, it was the first time that they had been asked to think about their data as ‘open data’ and progress was slower. For DataBC the project was not a compliance exercise, but an attempt to integrate governance, policy-making and citizen participation.
+
+### Project Resources (approximate):
+
+<table border="1" padding="1em">
+    <tr>
+        <td><strong>Phase,</strong></td>
+        <td><strong>Duration</strong></td>
+        <td><strong>Full-time human resources</strong></td>
+    </tr>
+    <tr>
+        <td>Planning and data-collection</td>
+        <td>3 months</td>
+        <td>3</td>
+    </tr>
+    <tr>
+        <td>Design and implementation</td>
+        <td>2 months</td>
+        <td>12-20</td>
+    </tr>
+    <tr>
+        <td>Ongoing</td>
+        <td>Since July 2011</td>
+        <td>8</td>
+    </tr>
+</table>
+
+### User comments and feedback:
+
+#### Herb Lainchbury, open systems developer:
+
+> “The main things I would like to see are:
+
+> * more raw data - right now in DataBC we see a lot of small aggregated data tables which are not all that useful,
+> * more interesting data - there are easy ways to tell what data is interesting but so far BC is not releasing much of it [...]
+> * license - I would also like to see them use a license that conforms to the open definition...”
+
+<p style="text-align: right">Thanks for input from David Wrate, DataBC</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter2-3">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter2-3.md

@@ -0,0 +1,91 @@
+---
+title: Introductions
+---
+
+## D-Brain in Korea: Digital budgeting and accounting system
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="no" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="no" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="no" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Country</th>
+        <td colspan="2">Republic of Korea</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://www.digitalbrain.go.kr">www.digitalbrain.go.kr</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Vital Statistics</th>
+        <td colspan="2">Total 77,000 registrations of central and local government officials and public agency users. 200,000 transactions by 15,000 users per day. Payment of about 3.6 trillion won ($3billion) on daily average (Korea’s 2010 budget : 292.8 trillion won). Connected to 55 related information systems.</td>
+    </tr>
+</table>
+
+<img alt="D-Brain" src="http://farm8.staticflickr.com/7100/7274122142_d489a01d58_o.png" class="screenshot" />
+
+The top-ranking e-Government nation according to the e-Government Development Index and UN Global [E-Government Survey](http://www2.unpan.org/egovkb/global_reports/12report.htm) was the Republic of Korea. Korea’s solution for a public facing Digital Budget & Accounting System (DBAS - or nicknamed D-brain), adopted in 2007, has also been making waves as a leading model of innovative digital budgeting.
+
+More than a data portal, D-Brain is a web-based participatory budgeting system, which ensures citizens’ participation throughout the entire budget cycle from budget preparation to audit. D-Brain was covered in detail by Gigler et al. (2011) in their report ‘Technologies for Transparency and Accountability: Implications for ICT Policy and Administration’ and as such, we will not replicate the same research. A couple of key points from their research are highlighted below.
+
+### Overview of Features (see [Gigler et al.](http://scr.bi/wb-report-draft))
+
+1. Integrated web-based system providing the public with real time analysis on government’s fiscal activities including budget formulation, execution, account settlement and performance management.
+2. Participatory budgeting where the central government, local governments, public institutions and the public collaboratively decide on the allocation of resources and participate in nationwide fiscal decision making.
+3. Citizen participation is enabled throughout the budgeting process through internet surveys, an online bulletin board, online bidding, a cyber forum, d-budget participation corner and public hearings to name but a few. Furthermore, there is a Budget Waste Report Center which offers a hotline and on-line system for citizens to prevent central government agencies and local government offices from misbehaving. Citizens are encouraged to report alleged misappropriations of government funds and are incentivised to do so with a ‘budget saving incentive bonus’, at most $30,000 which can be awarded to a citizen reporter if allegations are found to be true.
+
+### Benefits for Government
+
+1. System enables a better use of the national budget by reducing duplicative expenditure, leading the nation towards a more efficient fiscal policy. Oversight led by both policy makers and the public, who all have access to the necessary budget information to validate the accuracy and reliability of the budgeting records.
+2. Before the existence of d-Brain, Korea had experienced budgeting and accounting difficulties and no feedback mechanisms to monitor inefficiency. These difficulties included no ability to strategically distribute national resources, as information was not efficiently shared even within government departments.
+
+### Successes and Outcomes
+
+1. For the congress, it has become easier than ever to review budgeting and payment information for the different sub-ministries
+2. The budget authority is also able to make accurate budgeting decisions, due to their increased ability to review financial statements of previous projects in detail. They are able to better predict each expenditure line item of a future project, by which means it can also systematically manage the financial risk of the project.
+
+### Challenges
+
+1. The government needs to assess new areas that can be linked to the system and reflect it in the system to continuously maintain the efficiency of the system.
+2. Although the public participation rate has increased, it has shown that they have the tendency to remain as a passive user only making electronic payments and transfers. The public institutions will have to devise a way for them to become more active participants in the fiscal decision making process.
+
+### Scalability
+
+Some of the reasons for the success of the project are:
+
+1. ‘The nationwide ICT infrastructure and high ICT literacy’ - ICT training is subsidised by governments (Gigler et al.). Korea achieved the world’s second largest broadband penetration in early 2000 thanks, in part, to the active role of government in promoting it and according to the ‘Ten Million People Internet Education Project’, South Korea has the highest internet user rate as a percentage of the population. Computer literacy is also a requirement of entering college in South Korea.
+2. Web-participation is common in Korea, which provides most services over the Web. The approach may seem more alien and be less successful in countries where this is not the case.
+3. The initiative is supported by both the private and public sector. The Korean government steered the project, with cutting edge ICT technology provided by Samsung and LG CNS - this approach can be explored in other countries.
+4. D-Brain streamlines information exchange between central government, local government and public agencies, providing real time processing between agencies and facilitating payment collection.
+5. People are interested in fiscal activities, as more efficient use of funds by the government is directly linked to lowering their taxation.
+
+### Resources
+
+Total cost: $63 million
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter2-4">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter2-4.md

@@ -0,0 +1,101 @@
+---
+title: Brazilian Transparency Portal - Access to Information and Citizen Oversight
+---
+
+## Brazilian Transparency Portal: Access to Information and Citizen Oversight
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="no" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Country</th>
+        <td colspan="2">Brazil</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://www.portaltransparencia.gov.br">http://www.portaltransparencia.gov.br/</a></td>
+    </tr>
+</table>
+
+
+<img alt="Brazilian Transparency Portal" src="http://farm9.staticflickr.com/8007/7274151266_0ab15698f5_o.png" class="screenshot" />
+
+### Background
+
+
+In 2004, the Brazilian Office of the Comptroller General (CGU) created the Transparency Portal, a tool that aims at increasing fiscal transparency of the Brazilian Federal Government. Developed in partnership with the Federal Data Processing Service – SERPRO, the Transparency Portal relied on the collaboration of diverse Ministries and bodies of the Federal Public Administration to advance the transparency of information on the application of federal public resources and to offer a tool that can stimulate citizen participation. The Portal brings, in simple language, detailed information about how public resources are spent. It is known as one of the most important tools of e-government in regard to public spending control.
+
+### Available Fiscal Data
+
+#### Expenses:
+1. Daily Information: Budget and financial execution data with daily updating of the acts performed by administrators,
+2. Transfer of Resources: To states, municipalities, companies, transfers made in foreign land, or directly to an individual,
+3. Direct Expenses by the Federal Government: Construction contracts, government purchases, per diems paid and expenses on government credit cards, indicating the person that expensed any of the financial operations above
+
+#### Revenue:
+
+1. Budget and financial execution data with daily updates of the estimated government revenue.
+
+#### Other
+
+1. 2014 World Cup and Rio 2016 Olympic Games Expenses
+
+### Overview of Other Resources and Features
+
+1. Agreements: Register of Covenants, government partnerships  sealed in the past years
+2. CEIS: List of companies sanctioned by public entities
+3. Public Officials: List of public officials, including information about position, function and functional position, with a link to a schedule of remuneration and government positions
+4. Other links: List of all institutions of the federal government that have their own transparency websites
+5. Other information and Services: Guidelines about public participation and citizen control, including a section “saiba mais” (know more about it) about government programs and how to police/audit them,
+6. Download of queries
+7. Data in graphs and other visualizations
+
+### User Groups and Audiences
+
+The Portal registers an average of 410 000 monthly visits. The number of citizens accessing the Transparency Portal has grown from 10 000 to 336 512 per month, between 2004 and 2012. A total of 44 640 citizens are registered and receive information about resource transfer in government partnerships with 5 561 municipalities. Information is sent to registered users via email.
+
+### Successes and Outcomes
+
+There have been a number of concrete cases where the Transparency Portal has supported direct social control of government activities. At the beginning of 2008, Brazil’s domestic media published numerous reports about “suspect” expenditure made using federal government Payment Cards. In one case the reports led to the resignation of one federal minister. In other cases the portal’s data has given rise to unsubstantiated media reports.
+
+The Transparency Portal will only be successful if a variety of stakeholders are engaged in anti-corruption efforts. Citizens, non-governmental organizations, press, private sector, academic institutions, think tanks and other civil society actors have an indispensable role to achieve the project’s effectiveness. To improve the interaction between government and civil society, the Transparency Portal discloses a communication channel: “Contact us”. Through this channel, the Portal users can elucidate any doubts related to the accessibility or to its own content, as well as praise or make suggestions. Through this channel the Portal also receives many suggestions for improvement and denounces of misuse of public money.
+
+### Challenges
+
+Monitoring and evaluating the impact of the Transparency Portal is an evolving area. The Office of the Comptroller General of the Union measures the average time spent on the website, as well as its bounce rates, pages per visit, visitors, most demanded searches, among other data. Access numbers have been used to identify demands for different queries and the depth of information researched by users. This analysis allows web managers to prioritise the most requested information in the layout. Bounce rates can be used to improve e.g. the frequently asked question section.
+
+### Project Resources
+
+The Portal is administered by a Management Group and a Working Group (operational), composed of representatives of the areas of Corruption Prevention and Strategic Information, Internal Control and Information Systems, approximately 12 public officials (including Managers and operating team, most of them part time). In addition, there is a Technical Technology Group in total approximately 10 public officials working directly to Portal. This makes use of Public budget resources of the Federal Government.
+
+Approximately 137 000 USD was invested for the initial stage of the Portal’s development, from the Office of the Comptroller-General regular budget. Nowadays, the Office of the Comptroller-General of Brazil (CGU) controls the program’s database and the maintenance cost is relatively low.
+
+<p style="text-align: right">Thanks for input from Izabela Moreira Correa, CGU</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter3-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter2-intro.md

@@ -0,0 +1,26 @@
+---
+title: Chapter 2 - intro
+---
+
+## Chapter 2 - Publishing Fiscal Data: Government Perspectives
+
+<img alt="Uploading Data" src="http://farm8.staticflickr.com/7211/7285692458_bb8a771171_t.jpg" class="inline-image" />
+
+When it comes to proactive transparency and publishing fiscal data, there are numerous considerations to be taken into account. In this section, we take a look at different approaches to government financial data portals from the perspective of the people behind them; solutions range from out of the box, ready to implement, customisable, open-source solutions to custom sites, built specifically for the part of government they serve.
+
+We look at:
+
+1. Motivations for creating the service in the first instance
+2. Available fiscal data and features
+3. Successes and outcomes
+4. Challenges and areas for improvement
+5. Resources are needed to assemble and maintain such a site
+
+Featured Case Studies:
+
+1. [Data.Gov.Uk](../chapter2-1), United Kingdom
+2. [DataBC](../chapter2-2), Canada
+3. [DBrain](../chapter2-3), Republic of Korea
+4. [Brazilian Transparency Portal](../chapter2-4), Brazil
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter2-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter3-1.md

@@ -0,0 +1,66 @@
+---
+title: Introductions
+---
+
+## Testimonials: User Perspectives on using Government Fiscal Data
+
+The statements below all come from re-users of fiscal data outside of government. Each person talks about what fiscal data they required, what they wanted to do with it, and importantly, how easy it was to obtain and use
+
+### Canada: DataBC and tools for non-developers
+
+> “Another nice thing about the data.gov.bc.ca website is that it provides tools for non-developers, so that they can play with, and learn from, some of the data.”
+
+### Republic of Korea: Perspectives on D-Brain 
+
+> “D-Brain is only open to government officials viewing data. I cannot imagine getting data of my taste (say using SQL to select variables, and do simple analysis on it) from D-Brain. It only shows numbers of each categories by keyword searching.” 
+
+### Albania: Providing datasets to advocacy NGOs to challenge commonly accepted facts 
+
+> “In Albania, every governmental agency has to provide activity data, but the data is not being analytically used for different reasons. Sometimes the process of the data generation in these agencies is neglected, [...] and sometimes these data are erased after a short period. [...] The Albanian government has a statistical office, the Institute of Statistics (INSTAT), which works directly under the authority of Council of Ministers. There is a limited amount of the available data being published and the credibility of INSTAT is questionable - has been accused many times (on) of hiding important data or providing ambiguous data in favour of the government.” 
+
+### Australia: Monitoring industry influence on government 
+
+> Problems with the Data: 
+>
+> 1. Not openly licensed by default 
+>
+> 2. Not good data access - we have to get weekly exports every Sunday in CSV and after 2 years they disappear from the data store, so if a project was starting today they could not get all the data. There is no programmable API to access these exports, just a web page. 
+> 3. No validation/verification - having to build up the database week by week means we could lose transactions and not notice 
+> 4. Company data is closed - In Australia, the company register is pay-per-use so we cannot afford to find out who the directors of 300,000 different records are.
+
+### Bulgaria: Developing a visualisation of the Bulgarian budget using OpenSpending.org 
+
+> Interesting Documents: 
+>
+> 1. State Budget - Published as a PDF containing the budget law - very ugly. For the OpenSpending budget visualization I made, I had to parse the consolidated financial framework. [Level of detail is not sufficient] I would like to dig down to each expense. 
+> 2. Public Procurement Information - Published on a website - can be parsed. [Data is very timely] no more than a day or two delay. 
+> 3. State Audit Agency reports - Published on a website - can be parsed. Financial data on various topics of interest like: election financing, property of the public figures, political donations, party finances and other very, very interesting datasets. [Regarding timeliness] Periodic, they are ok with the terms. 
+> 4. Company register - Detailed information about all companies in Bulgaria way more detailed than is public in UK. [Unfortunately] Data is behind paywall - 15 000 euros/year, there is an API included in the same 15 000 euros/year cost. [Information is] very timely, updated every moment. 
+
+### Croatia: Institute of Public Finance 
+
+> “MoF ([Ministry of Finance](http://www.mfin.hr/)) web has a lot of interesting data [...] probably too rich, more fit for experts than for citizens or journalists. [...] [State Auditing Office](http://www. revizija.hr/hr/) claims that it puts all their reports on their web, but they don’t keep them there.” 
+
+### Estonia: Making budgets understandable for citizens 
+
+>“When published, the budget structure is too complicated, even for specialists in any policy field. The format is not machine-readable. So there is access to information but barriers for its usability and re-use by non-government groups.” 
+
+### Georgia: Increasing fact-based dialogue 
+
+>“One of our goals was to begin to make an online interactive interface which brings Tbilisi’s expenditures down to earth in a way a larger audience can understand.[...] The data, unfortunately, only covered the first two levels of the COFOG classification system, and so lacked the detail to allow me to understand in reality how Tbilisi City Hall spent funds. The reason I did not use a government transparency portal was because at that time it did not exist. Apparently, one exists now, but it was never advertised and nobody knew about it." 
+
+> “...it would make sense that the financial system used to create and implement Tbilisi City Hall’s budget and keep track of expenditures have a public API that would allow for real time queries that provided comprehensive, detailed, up-to-date and machine-readable results.” 
+
+### Greece: [Visualising the Greek Annual Budget](http://projects.thodoris.net/budget/ )
+
+> “We collected data by extracting them from PDFs [...] it would be ideal , if they made them available online in open - machine readable formats , either as digital files or through a web service. A similar problem occurred while we tried to collect financial figures for public debt and deficit” 
+
+### Mexico: Applied Research into public policy 
+
+> “The TP portal has visualizations and graphics that are useful to understand the budget process and it also has a citizen’s budget, but it lacks most of the raw [disaggregated] data you can find in the SHCP portal. The “Analíticos Presupuestarios” is a fairly good source of data but it lacks details such as the classification by “partida” (the most specific economic classification that the Mexican government use). It will be very useful if all the main budget documents have the same data structure (know it is not possible to follow some classifications in the in-year reports for example). Tools such as the [Peruvian “Transparencia Económica](http://ofi.mef.gob.pe/transparencia/Navegador/default.aspx)” are very useful to systematize specific information.” 
+
+### Nigeria: Building a budget-cut and crowd-sourcing oversight app 
+
+> “For sub-national data who have no defined government portal, we have been able to use our informal contacts to [get] data from two states out of thirty six states. We need to reach out to the states as most of them are not used to providing their budget data to the public. Our short term goal is to drive transparency within these states” 
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter4-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter3-intro.md

@@ -0,0 +1,28 @@
+---
+title: Using Fiscal Data - Civil Society Perspectives 
+---
+
+## Chapter 3 - Using Fiscal Data: Civil Society Perspectives 
+
+<img alt="User" src="http://farm8.staticflickr.com/7104/7286164510_2139fc2e89_t.jpg" class="inline-image" />
+
+The Open Budget Index has been instrumental in establishing the key budget documents that every government should publish. The Open Budget Index, grades governments according to their publication practices (timeliness, how many key documents are published etc.)<sup>[1]</sup>; the survey does not currently look at the file format of the released documents. 
+
+Getting format and content right is vital to ensure data can be used to hold governments to account. In this section, we examine from a user perspective how easy it is to find, get hold of and use that data. We asked the open spending data community (who come from a variety of backgrounds; research, technical, media etc.) what  fiscal data they required, what they wanted to do with it , and importantly, how easy it was to obtain and use. The examples demonstrate that simply publishing the data is not enough, attention must also be paid to how the data is published.
+
+Respondents were asked 6 questions: 
+
+1. How does data on government financial processes relate to your work? What is your mission? 
+2. How did you get the data? 
+3. If you used a government transparency portal to obtain the data, was it user friendly and were you able to find and access all the information you need? 
+4. If your government has a transparency portal but you did not use it - please explain why. 
+5. Please explain any issues with the data. 
+6. What could be done to make your work easier? 
+
+The [section below](../chapter3-1) contains quotes and in place paraphrased responses from some of the participants interviewed. 
+
+<div class='footnote'>
+    <sup>[1]</sup> Pre-Budget Statement, Executive’s Budget Proposal, Enacted Budget, Citizen’s Budget, (Supplementary budgets), In-Year Reports (Monthly / quarterly), Mid Year Review, Year End Report, Audit Reports.
+</div>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter3-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter4-1.md

@@ -0,0 +1,120 @@
+---
+title: IATI
+---
+
+## International Aid Transparency Initiative (IATI)
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="no" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">International</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://www.aidtransparency.net">http://www.aidtransparency.net</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Donors, parliaments of developing countries, private companies and foundations </td>
+    </tr>
+</table>
+
+<img alt="IATI" src="http://farm8.staticflickr.com/7100/7272476346_0ae81268a5_o.png" class="screenshot" />
+
+
+### Background
+Building on around 20 years of previous work, the first version of the International Aid Transparency Initiative (IATI) standard was launched in early 2011 and aid donors around the world started publishing to it. The IATI standard documents data about both aid donors and aid activities, enabling comparison and encouraging good practice in data management. The IATI standard also includes space to attach relevant documents and details of project results, to allow the standard to build context around the data and become an end-to-end solution, tracking projects from inception to execution.
+
+The IATI standard has seen far greater uptake than any previous effort in terms of organisations electing to publish their data in the standard. These successes are often attributed to the multi-stakeholder nature of the design process, including both policy makers and technical experts. From 2009 to 2010, IATI consulted with a wide range of stakeholders on the design of the technical standard, alongside a parallel process to secure donor support for publishing their aid information.
+
+### The Role of Technology
+
+In the past it was common to respond to information shortages by building a new database. But by working with open data principles, IATI allows a more distributed solution - where information can flow between organisations in many different ways, not just into a central database.
+
+The benefits of this approach have already been demonstrated, with many NGOs and charities choosing to follow the IATI Standard, although it was developed primarily with governments in mind.
+
+Donors publish aid information as a feed which can be read by many different applications; both those created by other donors, by the open data community, and - importantly - by software providers to developing country financial systems. By providing aid information in a standard format, many different users can access the data in the way they need to - and developing countries can see the resources, which are supposed to be flowing to them.
+
+### Main User Groups:
+
+1. Parliamentarians in developing countries gain a better oversight of the aid resources available. Knowing where to allocate resources in their own budgeting processes is vital to ensure that money is spent in the best way / most efficiently. Sometimes, there is a transparency-asymmetry between different parts of government. For example, a treasury may be very willing to open up the information it holds, but departments which benefit strongly from aid donations (e.g. departments of health) may be more reluctant to be transparent about aid revenues, as they will not want to 'lose out' from central government budgeting.
+2. Donors: Know where their money is going and whether it is being spent correctly.
+1. Private companies and foundations: e.g. [Akvo](http://www.akvo.org/), who are examining options for using IATI in their really simple reporting system (RSR).
+Aid Info Labs has also done a more extensive series of profiles on potential users of [IATI data](http://www.aidinfolabs.org/archives/category/inspiration/people).
+
+### Does the Project Require the Users to Have Specific Skills?
+
+Right now, direct access to raw IATI data requires some technical skills, but tools are being created like spreadsheet export of the raw data and preview tools to lower the barriers to entry.
+
+By working to improve the quality of the data and working with different infomediaries, from developers to researchers to activists on the ground, we can make sure that people can get access to the information they need.
+
+### Project Resources
+
+IATI centrally has needed both political and technical skills in the team - to facilitate a standard development process with a wide range of stakeholders. This involves being able to help people come together from across the world, which takes many resources.
+
+There are many different donors publishing IATI data, so they’ve needed different skills and support. Big donors have had to adapt their existing databases to output IATI data. Small donors have had a range of tools available to help them create IATI data - like openaidregister or aidstream.
+
+To use the data, some technical skills are needed, but also, importantly, skills to understand the complex aid infrastructure are important. AidInfo have been developing a one week course in understanding aid which will be piloted with NGOs in Nepal later this year, and which will be delivered alongside access to technical support to use IATI data provided by Young Innovations Nepal (YIPL).
+
+### Successes:
+
+1. The IATI registry now hosts links to hundreds of files of IATI standard data, covering thousands of aid activities. Over fifty percent of official Overseas Development Assistance will be covered by the IATI Standard now that commitments from major donors have been made to use the standard.
+2. Previously, it was difficult even for the governments of recipient countries to know how much money the country was getting from external sources. This affected their ability to be able to effectively prioritise budgeting “[U]nderstanding what donors are estimating to give to our country, whatever sector, helps us [...] to avoid what most ministries may do if we are not on top of information, doubledipping. They come to the Ministry of Finance and request for things that are funded by a donor.” (Brookings Institution Proceedings 2010).
+3. Proactive transparency: Lots of Charities (and not only those asked to do so by DFID) are looking at publishing IATI data, as the existence of the standard encourages more organisations to open up what they are doing. The drivers here may in part be around accountability, but also link to motivations of showing donors/public what the charities does, and finding opportunities for collaboration between organisations.
+4. The standard has made it worthwhile to build tools that make it easier to publish data in the correct format: such as [openaidregister](http://www.openaidregister.org/) and [Aidstream](http://aidstream.com/public/).
+5. The existence of this standard has sparked interest in mapping aid onto other types of financial data such as budgets. When budgets also conform to a standard, such as COFOG, tools can be built to do this work automatically. ‘Interfacing aid information with budgets needs to be tackled at the country level to make information relevant for a specific recipient country, and at donor headquarters level to facilitate the interface at country level. Country aid management systems need to be enhanced to deliver such functionality’ (Mills and Moon 2010).
+
+### Challenges
+
+1. There are still simple technical issues - not using the right XML field names, or the wrong format for dates. Initially this was a challenge, but the technical team have been developing tools to help assess data, and working with data publishers on data quality. This can't just be an automated process, as there are differences in data across the world which need human assessment.
+2. It is still difficult to cross-reference with other published figures such as budgets and expenditure (particularly when they do not balance).
+3. Adoption: While there have been many early uptakers, more work is needed to ensure that other donors and recipients are encouraged to implement the standard.
+
+### What is Needed to Address These Issues
+
+1. Quality-assurance tools to automatically check to see that a standard is being used well. Without these, downstream users of the data suffer technical problems.
+2. Grass-roots level trainers and community building: Working co-operatively with data publishers as far upstream as possible to check that the content of data is reliable. If this only happens when data comes to be used, lots of people have to duplicate the effort to make the data consistent.
+3. Data-management and merging systems: Publishers generating data in the IATI format might have data for different parts of the IATI Standard in different systems: e.g. project financials in one system, results in another, and mapping in another. If IATI data is generated from the projects system only, other content is missing from the IATI form. Having tools to allow merging of data, or allowing anyone to apply annotations to the data, would be valuable.
+4. Auxiliary information: A lot of information is locked up in the documents that IATI files can refer to (projects can have related documents), better linkage between them would be helpful.
+5. Tools for entity/concept extraction: Entity extraction tools allow the users to identify which individuals, organisations, projects etc. (i.e. entities) are present in a document, allowing them to understand relationships and which documents are related to which transactions and projects. If systems for visualising and presenting data throw the links to these documents away early on, then users’ transparency needs are not well served.
+6. The promotion of a standard for transactional level spending data could be useful. This would enable tools to be built which could automatically map aid data onto other budgetary information.
+
+AidInfo Labs documents plans, prototypes and products which make use of and build on IATI data. These can be found on their website, under the inspiration tab.
+
+### Learning Points:
+
+Getting organizations to share their aid data in a sustainable manner and to institutionalize the sharing of aid data through IATI takes time. IATI is already tremendously successful, but it is a long-term project and initial estimates number of donors publishing with IATI could be seen as somewhat optimistic.
+
+Adapting organisational processes to cope with change management,  and system adaptation to enable organisations to start publishing fluid, planning stage information, was difficult for organisations that were accustomed to more statistical reporting (DAC and CRS) and required more work than was initially expected.
+
+<p style="text-align: right">Thanks for input from Tim Davies, Michael Roberts and Mark Brough </p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter4-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter4-2.md

@@ -0,0 +1,44 @@
+---
+title: XBRL
+---
+
+## International Financial Reporting Standards: XBRL
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+</table>
+
+The eXtensible Business Reporting Language (XBRL) is a data standard for financial reporting, most commonly used to cover accounting information. The goal of XBRL is to efficiently create and disseminate data in a standardized, XML-based format that lends itself to [analytics](http://www.sec.gov/spotlight/xbrl/what-is-idata.shtml). XBRL makes few assumptions about the semantics of the data expressed in it, instead standardizing a framework in which specific reporting types, such as US GAAP or IFRS can be expressed as [taxonomies](http://www.xbrl.org/FRTApproved), each with a specific set of tags to express the reporting elements which are necessary to comply with the standard. A growing number of such taxonomies are developed and maintained by the [XBRL community](http://www.xbrl.org/FRTAcknowledged).
+
+XBRL has enjoyed widespread adoption as a reporting language for business accounting data across a number of countries, the most notable being the United States, where both the FDIC (2005) and the SEC (2009) have adopted the standard - by 2013 even international companies will have to file their reports to the SEC in XBRL/IRFS. There is little doubt that within a few years, XBRL will be the global lingua franca for accounting information. Going further, the [XBRL General Ledger](http://www.xbrl.org/GLTaxonomy) aims to make the standard usable within companies, storing detailed transactional and other operational information.
+
+Of course, this comprehensive commitment raises this question: If XBRL is accepted by governments worldwide for receiving data, shouldn’t it also be used to report their own finances? While there is some discussion on the use of XBRL for fiscal information, no taxonomies exist for budget information or transactional government spending. One concern regarding the use of XBRL as a reporting standard for government information may be its complexity: the XML schema requires the implementation of extensive standards to be interpreted. This would run contrary to the goal of transparency by raising the barriers of entry to the analysis of government financial releases. The use of common taxonomies, on the other hand, is a very desirable property, as could also be seen in the IATI example.
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter5-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter4-intro.md

@@ -0,0 +1,20 @@
+---
+title: Standards for Fiscal Data - towards an international framework 
+---
+
+## Chapter 4 - Standards for Fiscal Data: Towards an international framework 
+
+<img alt="Standards" src="http://farm8.staticflickr.com/7083/7285692382_03cbe2d4a3_t.jpg" class="inline-image" />
+
+Raw data is the fuel of many of the tools featured in this report. In later sections, we demonstrate that the poor quality of the available data and the fact that it is often in non-machine readable formats (e.g. PDF) can contribute considerably to the workload involved in developing these tools, or even render them entirely infeasible.’ 
+
+One of the recommendations of this report is to work towards a standard for transaction level spending data to enable greater reuse and comparison of, as well as confidence in, the data from a user perspective. 
+
+To demonstrate the impact that such an internationally recognised standard can have, the [International Aid Transparency Initiative standard is showcased here](../chapter4-1). We look at: 
+
+1. how a successful standard can be built with multi-stakeholder cooperation and 
+2. the possibilities this approach has opened up for aid transparency 
+
+We also briefly examine existing accounting standards which may inform how such a standard might look, focusing particularly on the [eXtensible Business Reporting Language](../chapter4-2).
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter4-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter5-1.md

@@ -0,0 +1,83 @@
+---
+title: Government of Sierra Leone GoSL - Online Repository for Ministry of Mines and Mineral Resources
+---
+
+## Government of Sierra Leone GoSL: Online Repository for Ministry of Mines and Mineral Resources
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="no" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Sierra Leone</td>
+    </tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://sierraleone.revenuesystems.org/">http://sierraleone.revenuesystems.org/</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Costs</th>
+        <td colspan="2">The mineral rights administrative system and the online repository have been implemented over a period of two years, costing approximately EUR 400,000 (including training, support and equipment for an administrative system)</td>
+    </tr>
+    <tr>
+        <th class="inner">Project executed by</th>
+        <td colspan="2">Revenue Development Foundation (RDF) </td>
+    </tr>
+</table>
+
+<img alt="GoSL" src="http://farm8.staticflickr.com/7239/7272473284_c043da6810_o.png" class="screenshot" />
+
+### Background
+
+A system that allows governments to publish all mining licences and associated revenues directly from their administrative system. The system can also present other types of revenue data such as forestry concessions, fishing rights, property and land rights.
+
+The system aims to increase government credibility and investor confidence in licenses issued by the government, and thereby increase foreign investments. The system currently has over 500 users, 65% of whom are mining companies and investors looking to verify government issued licenses and companies' good standing.
+
+Data comes directly from government systems and supports the Extractive Industries Transparency Initiative (EITI) by making audits of government receipts easier and accessible to the public. This generates a close accountability cycle between the public and government, as compared to EITI audits, where audits of government-received payments usually only takes place 2-3 years after they are made.
+
+The Sierra Leone government is in control of when and what to publish to the public and/or stakeholders, but typically publish updated data bi-monthly. The system contains all mining rights and license payments from 1 January 2010, as well as all exporters licenses and export taxes from 2008.
+
+Developers behind the site have attempted to make integration with existing government database management systems as simple as possible. Data is imported using an XML schema, which most database systems are compatible with.
+
+The online portal software can be provided for free and hosted by the Revenue Development Foundation to government institutions. However, it requires an administrative system to be in place from which the data can be extracted.
+
+### Challenges and Lessons
+
+Most government institutions in poor countries require extensive support to improve their administrative processes and systems in order to allow public scrutiny of their data. The GoSL Online Repository is the result of RDF support to the Ministry of Mines and Mineral Resources in Sierra Leone which has been given since 2009. RDF has seconded staff to their Mining Cadastre Office throughout the period, in order to change the administrative processes and improve data management. The Ministry use RDF's Minerals Cadastre Administration System (MCAS) to manage the mining licenses they issue, and the data is published to the online portal directly from this system.
+
+While the initial expectations were that civil society organisations would be prime users of the online data, experience has shown that mining companies and investors are the primary users, alongside law enforcement agencies.
+
+### Scalability and Future plans
+
+The system has been designed to be as flexible and scalable as possible. The backend can import a range of government license-and revenue-related datasets.
+The GoSL Online Repository was launched by the Government of Sierra Leone in January 2012. Similar systems are currently being deployed for Liberia and Gambia. Work is planned to start in Mali.
+
+<p style="text-align: right">Thanks to Aasmund Andersen, RDF</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter5-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter5-2.md

@@ -0,0 +1,109 @@
+---
+title: Revenues from Oil and Mining - Revenue Watch
+---
+
+## Revenues from Oil and Mining: Revenue Watch
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Global-level</td>
+    </tr>
+	<tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2">National governments</td>
+	</tr>
+	    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://www.revenuewatch.org/ ">http://www.revenuewatch.org/</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Civil society, national governments, international institutions, researchers</td>
+    </tr>
+</table>
+
+<img alt="RWI" src="http://farm9.staticflickr.com/8158/7272473162_226235031b_o.png" class="screenshot" />
+
+### Background
+
+New technologies and new trends in transparency have unlocked wide new streams of government data about oil and mineral wealth. But access to data does not guarantee access to knowledge. Like the raw materials that get converted to energy, disclosed information often needs refinement and a functioning infrastructure of expertise, analysis and advocacy before it becomes "combustible" fuel for change.
+
+Revenue Watch tools are aimed at helping to "follow the money" and make sure that wealth in the ground translates into development and economic growth above the ground. RWI and partners work to make the management of resource wealth more responsible, more transparent, and easier to understand. New technology is an increasingly vital part of this effort.
+
+Revenue Watch has spent more than seven years promoting access to information as an indispensable tool for improving resource management for the public good. Over the last two years, they have been working to define and pilot effective uses of web technology for the display, analysis and dissemination of oil, gas and mining governance data.
+
+### An Overview of Tools and the Role of Technology
+
+The [EITI report analysis tool](http://data.revenuewatch.org/eiti) is a tool built by Revenue Watch based on information extracted from over 50 national reports from the Extractive Industries Transparency Initiative (EITI), a voluntary standard in which governments, civil society and companies work together to report payments and government receipts from oil, gas and mining. RWI’s tool presents per-country analysis using RWI's own indicators of report quality, features for comparison of different country results, and easy tools for sharing and downloading data.
+
+To demonstrate the significance of disclosure rules for international oil and mining companies, RWI launched an [interactive calculator](http://data.revenuewatch.org/listings) illustrating the number and value of extractive sector companies listed on the largest global stock exchanges. This research and the streamlined presentation by value, by exchange and by sector, allows non-experts to explore the data themselves, and better understand the high stakes in new US and proposed EU laws requiring companies to come clean about what they are paying to governments of resource-rich nations.
+
+A core principle of Revenue Watch’s technical work is that gaps in strategy or in familiarity with online modalities are the main obstacles to better uses of tech for transparency, as opposed to gaps in technology investment. Revenue Watch believe that by developing not only digital tools, but also replicable approaches to data distribution, usability and user training, they can set a higher standard for the use of oil, gas and mining information by governments, companies, advocates, journalists and citizens.
+
+In addition to these two data tools launched during 2011, Revenue Watch also recently created a simple, highly visual interactive for comparing oil company profits with oil producing economies, at the self-explanatory web address [OilvsWorld.com](http://www.revenuewatch.org/issues/dodd-frank/oilvsworld) (as well as OilvsAfrica.com, OilvsEurope.com, OilvsAsia.com, OilvsAmericas.com).
+
+The [Revenue Watch Index](http://www.revenuewatch.org/rwindex), a pioneering measure of oil and mining disclosure practices in more than 40 countries, was launched in 2010 as a way to break down and assess transparency practices. The index, created in partnership with Transparency International, is an assessment and comparison of information published by governments about revenues, oil savings funds, sovereign wealth funds and state-owned enterprises, contract terms and other key data. It is an important tool for elected officials, policy makers, civil society and the media when seeking increased public disclosure about natural resource management and greater government accountability. The 2012 index will feature a much richer set of underlying data which will be available for breakdown and comparison by region, country and indicator. This new version of the index is supported by Global Integrity's research platform [Indaba](http://www.getindaba.org).
+
+Revenue Watch is also currently collaborating with the Open Knowledge Foundation (OKFN) to release an interactive database of original research compiling fiscal rules and royalty practices for mining in African countries.
+
+### Specific User Skills and Usability
+
+To maximize not just usability, but “use” of the tools, Revenue Watch worked on design for less-web-savvy users, working in lower-bandwidth settings, possibly on older computers. To maximize dissemination across social networks, where personal connections ensure information shared is more relevant and thus more persuasive, Revenue Watch paid particular attention to tools for sharing the data, including custom links to share user-created views (building on the examples of the World Bank and Google, among others). And, given the reality that the most effective advocacy and analysis often happens "offline", particular emphasis was given to making custom views and full data sets easier to download or print for later use - particularly in the case of the EITI data.
+
+As new tech approaches are adopted and integrated further across RWI programming, their tools and advocacy will prioritize easier adoption of technology and smarter integration of tech by all actors in the natural resource governance sector.
+
+### Successes
+
+RWI's analysis of EITI reports and the EITI data tool represent the first ever compilation of the numerous country level reports in one place and extraction of the report data into a sortable online database. Taken together, these tools helped to demonstrate the value of EITI reporting and provide the first one-stop destination for this data for users in the field, including, for instance, officials at the International Monetary Fund, who decided to use the tool as a resource rather than building their own EITI repository. EITI Chair Clare Short has often cited RWI's research and report tool since it debuted at EITI's 2011 global conference. Most importantly, the RWI tool has served as a model for EITI undertakes its own plan to harvest report data and offer the data online.
+
+### Issues with the Data and Standardisation
+
+One challenge within the EITI data has been issues of comparability and standardisation. RWI had to carry out additional calculations and make certain decisions in order to enable more meaningful comparisons to be drawn from the data. This is inevitable given the nature of EITI reporting itself.
+
+One challenge especially illustrative of the difficulties of creating data sets for oil, gas and mining governance came whilst RWI were carrying out research into oil and mining companies on the largest international stock exchanges. A widespread practice by companies of listing the same securities on multiple exchanges made it difficult to accurately calculate the cumulative market capitalization of oil and mining companies across multiple exchanges. Even the data provider most commonly used as a source for securities data (Bloomberg) turned out to have data that was still full of "cross-listings" of the same securities. "Cleaning" the data to approach a more accurate estimate of market capitalization (a core statistic for the tool's presentation of oil and mining sector value) was a labor-intensive task that, while replicable, was not one that automation could solve due to the level of "noise" in the data. This basic statistic - How much is the sector worth on a given exchange? - is itself very difficult to determine. This is taken as a further sign that better practices in the industry are need in order to reach effective fiscal transparency.
+
+A good dataset does not guarantee a good database or tool. Most importantly, even a working tool that offers simple access and downloadability of the data does not guarantee usability or "understandability" of the data. This step is the one most often short-changed in the open data activity cycle. Data is not always actionable simply because it is available. A simple interface designed according to user-centric principles, with tools that give the end user power to use the data according to their needs, and which has context and guidance to show what the data means instead of just what it ”is”, are all fundamental elements of the broader "accessibility" that make data leverageable for greater knowledge and further advocacy.
+
+### Project resources
+
+The two main data sites mentioned above required several steps to build: data collection, data analysis, tool creation and interface design. A total of only 4-5 people were involved across all phases, but it is important to note that the skill sets required for each of these elements are quite different. Good data collection does not guarantee good data modeling for comparability (or deduplication, as in the stock calculator example).
+
+### User comments:
+
+> “Of the sources you mention the only one I have regularly used is Revenue Watch's Stock Exchange Calculator. It's been very useful for comms purposes (as a place to source a fact/figure to support my [argument](http://publishwhatyoupay.org/newsroom/blog/why-dodd-frank-1504-won%E2%80%99t-undermine-competitiveness))”.
+
+
+>“The Oil vs the World tool is good but could be even more useful with a different set of indicators (I'm not sure about the comparability (even if for the sake of the effect of contrast) between GDP and oil company profit. More effective could be to see revenues generated per country for their natural resources vs population living on less than 1 dollar; or oil company profit vs cost of compliance etc...)”.
+
+<p style="text-align: right">Thanks to Jed Miller, RWI</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter5-intro.md

@@ -0,0 +1,17 @@
+---
+title: Case Studies - Where Does the Money Come From? 
+---
+
+## Chapter 5 - Case Studies : Where Does the Money Come From? 
+
+<img alt="Revenue" src="http://www.flickr.com/photos/okfn/7285692280/" class="inline-image" />
+
+In order to assess how much money is available to spend, a government must know how much money it receives. Governments receive revenues in many ways, ranging from taxation to income from natural resources. This section examines tools for transparency regarding revenue and the processes around extractive industries. Transparency around taxation policies is covered in the section on participatory budgeting. 
+
+
+Case studies include: 
+
+1. [Government of Sierra Leone GoSL](../chapter5-1): Online Repository for Ministry of Mines and Mineral Resources 
+2. [Revenues from Oil and Mining](../chapter5-2): Revenue Watch
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter5-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter6-1.md

@@ -0,0 +1,85 @@
+---
+Title: ClearSpending from the Sunlight Foundation
+---
+
+## ClearSpending from the Sunlight Foundation
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="no" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="no" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">USA</td>
+    </tr>
+	<tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2">Federal level</td>
+	</tr>
+	    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://sunlightfoundation.com/clearspending/">http://sunlightfoundation.com/clearspending/</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Civil society, national governments, international institutions, researchers</td>
+    </tr>
+</table>
+
+
+<img alt="Clearspending" src="http://farm8.staticflickr.com/7226/7272476246_5ba1cbc2e7_o.png" class="screenshot" />
+
+### Background
+
+Clearspending analyzes the data quality of the grants data in USASpending.gov, the cornerstone of President Obama's transparency platform even before he took office. He co-sponsored the bill that created it whilst he was a Senator, and oversaw part of its execution in the executive branch. Clearspending addresses three metrics: timeliness, completeness, and consistency with other government estimates. Over $1.3 trillion failed on at least one of these metrics. Users of Clearspending can view topline data quality measurements by agency, or drill down to a program level to view the performance of specific grant and loan programs. Essentially, Clearspending is an oversight tool that examines the compliance of each agency and program with grant and loan reporting requirements. The site also provides substantial background information on the evolution of the reporting systems that power it (some are decades old) and the overall picture of how spending is reported in different areas of the federal government. The original methodology of Clearspending stems from a Government Accountability Office report on the same topic, but the sample-based methodology has been expanded with an automated program to examine all transactions instead of only a sample percentage.
+
+### Target users
+
+The targeted users are anybody who uses USASpending.gov to get spending data on grant and loan programs. This usually includes, but is not limited to, journalists, academics, policy analysts, concerned citizens, and congressional staffers. For anyone looking at a particular grant or loan program, Clearspending can tell you how much of the program's actual spending data you can expect to find in USASpending.gov. You can also see how on time a particular program or agency is in reporting their spending. Many agencies and programs report far past the 30 day statutory requirement, so their spending can vary widely depending on when you view the data, even after the given fiscal year has passed. As an auxiliary use, Clearspending is also designed to be used by anyone who would want to investigate which agencies or programs are not fulfilling their legal mandate to publish spending information online in a complete and timely manner.
+
+### Successes
+
+The project saw considerable public success as an oversight tool. The House of Representatives Committee on Oversight and Government Reform convened two hearings where the Executive Director of the Sunlight Foundation, Ellen Miller, was invited to testify regarding Clearspending analysis, one on the subject of achieving fiscal transparency and one on evaluating open government initiatives. Agency CTOs with particularly poor examples of data quality were also invited to testify. Follow-up questions and reports from the Committee resulted in the correction of some of the most notable errors.
+
+Most of the errors that were pointed out were related to non-reporting programs. Specifically, the Department of Agriculture had chosen not to report for several programs because they had misinterpreted the guidance. They later wrote to the Oversight Committee explaining this, and pledging to begin reporting the spending. Also, after demoing the beta version of Clearspending to the department of Health and Human services and showing them the gap in medicare/medicaid reporting that Clearspending highlighted, they began back reporting Medicare data.
+
+### Challenges
+
+The main challenges of the site stemmed from the complicated picture of the federal spending systems themselves. Sketching a broad view of how the system is designed to work and at what points it is failing proved difficult for most casual users, and even some very knowledgeable users. Federal spending data is nearly impossible to simplify without losing a good deal of information that is valuable to experts on the subject.
+
+### Data Problems
+
+The major problems with the data were programs that, for whatever reason, chose not to report their spending. Additionally, many programs reported their spending long after the 30 day window in which they were supposed to, and sometimes even an entire fiscal year later. Another counterintuitive problem was the issue of over reporting. Confusion on the reporting guidance led to student loans being reported in the $6-7 trillion range, instead of the actual $60-80 billion range.
+
+### Future Plans
+
+The Sunlight Foundation re-released Clearspending with new data in Fall of 2011, and there are plans to update the analysis each year with new data. More ambitious aims include expanding the analysis to contracting data, which is a more difficult task, due to the lack of the kind of comparison data that was present for grants. Several FOIA requests for information regarding contracts data were in progress at the time of the report.
+
+<p style="text-align: right">Thanks to Kaitlin Lee</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter6-2.md

@@ -0,0 +1,43 @@
+---
+title: The Media and Fiscal Data - How do we get fiscal data into the public eye?
+
+---
+
+## The Media and Fiscal Data: How do we get fiscal data into the public eye?
+
+![Europe's Missing Millions from the Bureau of Investigative Journalism](http://farm9.staticflickr.com/8157/7295526880_5f5db8d268_o.png)
+
+
+One of the key promises of open data is that it can be used by journalists to get a clear, evidence-based picture of government action, in particular by using fiscal information to track budgetary priorities, contractual commitments and patterns of spending. As part of the OpenSpending project, the Open Knowledge Foundation is working closely with journalists to develop tools which aim to make financial data released by governments accessible and usable in a journalistic context. We are regularly approached by journalists asking for advice on where to find information on a particular aspect of government spending. In this section, we highlight a couple of our experiences from building these tools, and document some anecdotes from other organisations who have had varying degrees of success with reaching out to the media.
+
+As a part of this project (Spending Stories), we identified the following challenges:
+
+1. Journalists are often not used to working with raw data, and don’t consider it a necessary foundation for their reporting. Sourcing stories from raw information is still a relatively new idea.
+2. They also often do not possess the necessary literacy and technical skill to analyse the data and present their findings. This includes technical skill, but also statistical and design capacity.
+3. Analyzing and understanding data is a time-intensive process, even with the necessary skills. Fitting this into a short-lived news cycle is hard, so data journalism is mostly used in longer-term, investigative projects. This is reflected in a statement by the Budapest Institute in Hungary: “One of the conclusions of the workshop with journalists and the following consultations with media representatives was that our initial idea [of building a tool that would be useful for journalists] turns to be a false hope. [...] We have learned that the Hungarian media is prone to get ready-made analytics and reports rather than to perform investigative projects on its own.”
+4. The data released by governments is often incomplete or not up-to-date, making it harder to argue a story based on the available information. Very often, released databases cannot be used for investigative purposes without the addition of FOI-requested, more specific pieces of information.  In an article in the [Guardian Datablog](http://www.guardian.co.uk/society/datablog/2011/mar/18/public-finance-data-store) which followed the release of the UK Local Council spending data, Lisa Evans explained why, despite the impressive show of proactive transparency from the UK government, many barriers still prevent journalists from using the data to hold the government to account. The issues she identified include concerns about the completeness of the data; for example, the inability to see money which is allocated in [Private Finance Initiatives](http://www.guardian.co.uk/politics/pfi) (Public-Private Partnerships).
+
+A couple of observations on Spending Stories as a tool:
+
+1. We decided early on that we would not confine ourselves to providing journalists with technological solutions, but would also offer training sessions in the use of data and relevant tools. Feedback from participants indicates that this method training was useful. It has also helped us to gauge the level of skill which we can expect users of our technological solutions to have.
+2. We started under the assumption that our tool would be aimed primarily at journalists, but have found it is more commonly used by advocacy groups, who have more time to dedicate to research. These advocacy groups then contact journalists with the story in order to gain publicity for the topic once an outline of the story has been researched and the bulk of the data work has been completed.
+
+When we explored the specific information- and tool-requirements for journalists who wanted to work with spending data, the following needs were recurrent:
+
+1. Heuristics for story-finding: This could include the automated calculation of statistical measures such as percentiles, scatterplots, standard deviations - but also very specific analysis such as measuring the size of firms in relation to the industry (Herfindahl- Hirschman Index).
+2. Time-series visualisations of total spend to date by quarter: This would allow journalists to detect problematic practices such as those highlighted by Vivek Ramkumar’s report: “if the majority of expenses are incurred in the last quarter of the year, this could indicate that the agency was keen to spend money even if it meant wasting it so that it could apply for the next installment of funding in the subsequent year.”
+3. Notification services for journalists: When an interesting dataset is published, journalists would get a notification directly to their inbox. When data is open, the determining factor is not who gets exclusive data, but who has the skills to quickly analyse and interpret the data.
+4. Search options - how to check whether the datasets (or similar datasets) have been published anywhere)? Some journalists also asked for tools to identify hotspots - to let them see what the most common searches were so that they could get an idea of the topics readers were most interested in and target their stories around those.
+5. Ability to call up structured data around the budgeting process is very important: Such information is required in order to follow debates and to see how different MPs voted, as well as to follow amendments, when appropriations were changed, and who was responsible for making the changes. This also includes easy reference to e.g. laws which affect certain spending patterns laws which affect certain spending patterns, ideally presented in a handbook or short glossary format.
+6. Linking the numbers to people and organisations: Numbers can be offputting - journalists need to put faces to them in order to make them accessible. Other requests in this vein were a personality tracker - people are particularly interested in following particular high profile names, so it would be interesting if you could follow particular people. For example, having information which lets journalists say ‘here are all the laws undersigned by a particular politician- are there any patterns’?
+7. Disaggregated data or more information to understand provenance of aggregates: What is the source of the data? What is the formula via which the end result is reached?
+
+### Read more:
+
+* <http://blog.openspending.org/2011/10/27/thoughts-from-the-global-investigative-journalism-conference/>
+* <http://www.guardian.co.uk/society/datablog/2011/mar/18/public-finance-data-store>
+* <http://www.guardian.co.uk/news/datablog/2012/mar/16/us-open-spending-data>
+* [Sunlight Foundation: Reporting group](http://reporting.sunlightfoundation.com/SLRG/)
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-3">Next &raquo;</a></div>
+

examples/openspending/content/resources/gift/chapter6-3.md

@@ -0,0 +1,135 @@
+---
+title: Deciding where the money goes - Participatory Budgeting
+---
+
+## Deciding where the money goes: Participatory Budgeting
+
+### e-participatory budgeting
+
+Participatory budgeting (PB), put simply, gives citizens direct input into how government money is spent, for example via budget consultations or in-person meetings. PB occurs in over 1500 cities over the world, and there is a [Google map](http://bit.ly/IS8Skf), which shows (marked with red pointers) locations which have favoured a technical solution, referred to here as ‘e-PB’.
+
+For government officials interested in e-PB, the Participatory Budgeting Unit have produced a useful paper including a list of considerations which governments [should take into account](http://bit.ly/PB-role_of_technology).
+The collective knowledge of the [Participatory Budgeting Google Group](http://groups.google.com/group/participatorybudgeting?pli=1_) is another useful resource for anyone seeking advice on implementing PB, using technical methods or otherwise. Many members of the group develop e-PB systems, and the group has built a [census](http://bit.ly/PB-software-census) of existing technical solutions.
+
+The census shows that there are a large number of competing software solutions (both proprietary and open-source) but no real consensus on a particular approach to implementation. When it comes to building a software tool for a process as complex as participatory budgeting, perhaps one size does not fit all. In what follows, we have sought to determine activities in the PB process for which the projects in the census may be able to provide solutions.
+
+At the end of this section, we present three in-depth case studies of different technical solutions.
+
+### PB Activities targeted by technology
+
+The following bullet points highlight key activities in the PB cycle and outcomes which some of the case studies highlighted in this section seek to target:
+
+1. **Outreach**: Encouraging the ‘hard to reach’ to participate, typically the disenfranchised, the busy and the apathetic. Targeted outreach is also important, being able to contact people who will be most affected by proposed policies or projects.
+2. **Following projects and collecting feedback**: The ‘hard to reach’ could equally be defined as finance officers and decision makers in government who have little in-person contact with their citizens. Technology can offer them a way to connect with their citizens.
+3. **Publicising progress**: To demonstrate clearly what effects the participatory voting system is having, e.g. which projects were funded in previous years by using this system, how many participants etc.
+4. **Aiding deliberation and facilitating debate**: Structuring arguments and collecting ideas e.g. for potential new projects.
+5. **Remote participation**: Some people who are unable or unwilling to participate in person may be willing to do so online or via mobile.
+6. **Reducing costs**: Commonly incurred costs during a PB process are outreach, planning, running the meetings and collecting the feedback.
+7. **Reducing workload**: Running the process can put a lot of stress on staff particularly around planning in the run up to and follow-up from meetings. In offline participatory budgeting, substantial amounts of outreach work and tasks such as translation for minority groups are often taken on by volunteers.
+8. **Raising additional funds**: Governments are frequently strapped for the money to implement projects. If additional funds are required to get a project off the ground, some projects highlighted in this section turn to  citizens or businesses to make in-kind or cash contributions to get the project off the ground.
+9. **Increasing budget literacy**: Simulators allow participants to explore how certain spending/revenue choices impact the budget and gain an appreciation for how much state projects cost. This allows citizens to explore trade-offs between different options.
+
+### Possible solutions
+
+This section highlights existing initiatives which take advantage of the opportunities highlighted in the section ‘PB Activities targeted by technology’ .
+
+
+#### Outreach
+
+**Using technology to invite people to take part in offline voting**
+
+*  e.g. geo-targeted SMS (See the Mobile-Enhanced PB case study, where, when asked how they had heard about the participatory budgeting meeting, over 50% of respondents cited the SMS invites they received as the reason),
+*  automatic phone-calls where the voice of the mayor encourages people to come out to vote (good for illiterate populations, and hearing the voice of an authority figure may help convince people to participate),
+
+**Combining online and offline approaches**
+
+1. e.g. putting e-voting stations outside churches and other places where large numbers of people congregate.
+
+
+#### Following projects and providing feedback
+
+<img alt="Gol App" src="http://farm8.staticflickr.com/7100/7272472352_2724487244_o.png" class="screenshot inline-image" />
+
+**‘Problem-solving approaches’**
+
+*  e.g. [Fix My Street](http://www.fixmystreet.com/) / [SeeClickFix](http://makehoustongreat.com/seeclickfix/) or Lichfield council’s [Fix My Tweet](http://www.fixmytweet.com/) - where citizens identify tangible problems e.g. potholes on the street and submit a request via their mobile / via Twitter to their local authority to have them fixed.
+
+**Following the progress of projects**
+
+*  e.g. [The Gol Mobile App](http://itunes.apple.com/us/app/porto-alegre/id479880903?mt=8) for Porto Allegre is a good example of an application which allows users to track the progress of a request and submit progress reports via their smartphone.
+
+#### Publicising progress
+
+*  Publicising information on the outcomes of previous rounds of participatory budgeting and what has become of the projects.
+*  [Bürgerhaushalt Lichtenberg](http://bit.ly/yE7xGa) publishes updates on projects suggested by citizens, including implementation stage, relevant authorities overseeing the work and other comments regarding the projects.
+
+#### Deliberation
+
+**Online deliberation:**
+
+*  Many projects allow citizens to submit their own project ideas. e.g.: The UK’s YourLocalBudget project, [‘Budget Ballot’](http://budgetballot.com/)
+*  Allowing direct feedback into laws already in draft stages. e.g. [Adhocracy](https://adhocracy.de/), which allows users to cut, add to, reword and restructure proposed bills
+*  Structuring argument. Example: The [MIT deliberatorium](http://cci.mit.edu/klein/deliberatorium.html), which requires users to ‘map’ their argument e.g. pro-contra a point made by another, follow-on questions, ideas for solutions to previously raised problems.
+
+**Offline deliberation, aided by tools.**
+
+*  Example: ‘America Speaks 21st century town-meetings’ are in-person meetings which use computers on every table to serve as “electronic flipcharts” to record general table agreements; table agreements are instantly transmitted to a “Theme Team”, which reads who read all the entries to identify the strongest themes. These overarching themes are displayed and quickly presented to all the participants; individuals use their individual voting keypads to vote on what they believe are the most important priorities.
+
+#### Remote participation:
+
+*  online voting and surveys (numerous examples in census. See also the Citizen Budget case study).
+
+#### Cost reduction:
+
+*  Outreach e.g. advertising meetings on social media
+*  Email - usually cheaper than paper outreach.
+*  Even SMS can be a relatively cost effective option. In the Dominican Republic, the World Bank negotiated with phone companies a rate of around $0.01 per message. A single callout on radio cost around $200 to advertise the meeting. Nonetheless, text messages appeared to be a more effective option in terms of the number of people motivated to join the meeting (see statistics in the Mobile-Enhance Participatory Budgeting case study).
+
+#### Reduction of workload
+
+* Systems which help to manage and direct volunteer efforts: e.g. collaborative editors, (Google Docs), Microtask management (Tasket), Translation ([Amara](http://www.universalsubtitles.org/en/), [Transifex](https://www.transifex.net/)).
+
+#### Raising additional funds
+
+* Very few [Kickstarter](http://www.kickstarter.com/)/[Pledgebank](http://www.pledgebank.com/)- style applications exist for publicly funded projects, however this approach has been incredibly successful in crowdsourcing support for non-governmental projects via the internet. One of the few attempts to do something similar is: [Leih Deiner Stadt Geld](https://www.leihdeinerstadtgeld.de/) in Germany, this works by encouraging people to invest in their city.
+
+#### Increasing Budget Literacy
+
+* Budget simulators: The Estonian site [Meieraha.eu](http://meieraha.eu/) allows people to visualise and explore the effect of different revenue and expenditure policies e.g. raising taxes.
+* Personal Impact Calculators: One aspect which very few of the budget simulators dealt with was personal impact, where users are shown what the impact on them personally would be, e.g the effect that increasing spending above budget levels would have on the amount of tax that they as an individual would pay. Large numbers can be disorienting for the average citizen who may find it difficult to relate the numbers to more familiar measurements such as household budgets and their daily salary. An example attempting to solve this problem is [Budget Allocator](http://demo.budgetallocator.com/), which explains to users the impact of their choices: “Although you have balanced your budget, based on your current selections, we may still have to increase rates by 6.4%. Typically a 1% rates increase equates to 12 cents a week.”
+* Research is currently in progress at the Ash Centre in Harvard into forcing people to watch an explanatory video before voting. This will obviously not work in all situations (e.g. places with bandwidth issues), however it does have the benefit over text that it ensures that information cannot be ‘skipped’.
+
+### Concerns about using technology in PB
+
+The concerns below were voiced about using technology in PB by those both inside and outside governments:
+
+* “Possibilities for gaming the system” (e.g. with bad voting systems).
+*  ‘Herding effects’ from special interest groups, which could be difficult to monitor if participation is remote.
+*  Using technology as the ‘easy option’ and not dedicating enough time to face-to-face interaction.
+* For both citizens and governments: Not having the right skills to be able to deal with such a system.
+* Creating additional work for government officials by allowing torrents of unstructured, unmoderated and often duplicate comments to flood in, all expecting answers.
+* Information cascades in online voting systems: Due to the volume of responses, participants often do not have time to review all of the proposed solutions, so if only the most recent comments and suggestions appear at the top, or only top ranked, they may gravitate towards selecting these.
+* Who participates? Particularly, where remote participation is concerned: Ensuring that only relevant constituents are participating.
+* Finding the right balance between a simple and usable application and an oversimplified solution.
+
+### e-PB: What can policymakers do to help projects such as these succeed?
+
+This section has dealt mainly with tools for direct participation, as opposed to indirect participation, where citizens choose their preferred policies via the proxy of an elected representative. In the latter case, there are also ways in which governments can help to promote informed decision-making in electing their preferred candidate. For example, by releasing structured information on voting records of a given candidate.  In countries such as Kenya, where individual politicians hold sway over a large discretionary fund (Constituency Development Funds), having access to information on an individual’s previous financial behaviour can be crucial to selecting the correct representative. See [this post](http://openspending.org/blog/2012/01/30/hakuna-my-data-nbo-data-bootcamp.html) on ‘Auto-generated campaign speeches’ for one suggestion of a project to compare candidates based on their financial track-record.
+
+Many of the projects in this section are civil society led and are listed here to provide inspiration for some of the ways in which technology may be used to promote greater citizen participation around budgeting issues. However, these efforts can only go so far without support from governments. The following section highlights a couple of ways in which governments could promote such projects as these.
+
+#### Promoting public participation in the annual budget cycle, and in the design and delivery of public services and public investment projects
+
+1. Publish machine-readable schedules of key events in the budget cycle e.g. publication of key documents, consultation periods etc.
+2. Make sure teams within governments have the resources to deal with potentially large volumes of feedback in order not to disappoint those in their constituencies by not being able to take suggestions into account.
+3. Make sure technology does not become a substitute for face-to-face contact. Many important parts of the PB process happen offline. Technology is a useful complement to offline channels of communication and participation, but is not a substitute.
+
+#### Promoting participation around legislation concerning budgeting issues
+
+1. [Examine structured data formats for drafting legislation and tabling amendments](http://www.publicwhip.org.uk/project/liaison.php).
+2. Be open and transparent about [how legislation is drafted](http://www.ictparliament.org/xmltraining_brussels2012). Progress was made recently when the European Parliament (EP) decided to [open-source At4am](http://bit.ly/KhOr13), software that helps staff at the EP write and table amendments. Such moves help those developing tools for deliberation and collaboration around ideas outside government understand workflows and adapt their tools to work with existing government workflows.
+3. Note that while the European Parliament plans to release the At4am software, [it has not currently announced a plan to make the data from its own copy available](http://www.asktheeu.org/en/request/source_code_and_data_for_at4am_s_2), and doing so would represent a substantial increase in transparency around legislators’ activities.
+4. Structured information on who in a department is responsible for [particular changes](http://pudo.org/2011/07/22/kleine-charta-der-beteiligten.html). It is unproductive to attempt to talk to an entire department about a proposed project or change, who are the people who can actually make a difference?
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-4">Next &raquo;</a></div>
+

examples/openspending/content/resources/gift/chapter6-4.md

@@ -0,0 +1,122 @@
+---
+title: Mini atlases for participatory budgeting - Solo Kota Kita (Solo, Our City)
+---
+
+## Mini atlases for participatory budgeting: Solo Kota Kita (English: Solo, Our City)
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="no" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Indonesia</td>
+    </tr>
+	<tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2">Local government</td>
+	</tr>
+	    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="<http://solokotakita.org/en/"><http://solokotakita.org/en/</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Donors, recipient country governments, citizens, NGOs</td>
+    </tr>
+</table>
+
+![SKK Image](http://farm8.staticflickr.com/7231/7295628522_e2dce3fc25_o.png)
+
+### Background
+
+In 2001, Indonesia underwent ambitious decentralization programs. The "big bang" reforms devolved political, administrative and economic power to smaller administrative units, and local actors took over management of municipal infrastructure and systems. Against this backdrop, the national as well as local governments developed and supported new planning processes that sought to increase community participation in government decision-making.
+
+Participatory budgeting (known in Indonesia as Musrenbang) was piloted by a group of NGOs in Solo for the first time in 2001. In 2005 the national government mandated that every region and city in Indonesia implement an annual Musrenbang process. As part of the Musrenbang process, residents in a city or region meet to prioritise and propose short-term improvements they'd like local government to implement in their neighbourhoods.
+
+While participatory processes such as Musrenbang are in place, in many cases local authorities and communities lack capacity or tools to maximize the collaborative decision-making opportunities these processes offer. For example, priorities set during Musrenbang meetings don't always correspond to the most urgent requirements of a given neighbourhood. SKK’s first project sought to tackle this issue in Solo by increasing awareness of urban issues to strengthen residents’ voices and their capacity to prioritise community issues during the Musrenbang process.
+
+### Project Description
+
+The tool, ‘the Mini-Atlas’, presents a map and thematic information about each neighbourhood in Solo. Each atlas includes a map of community facilities (responding to local feedback that often the participatory budgeting meetings happen without a map of the neighbourhood) and thematic data regarding education, water, sanitation, housing, poverty, and health, highlighting key assets and issues for residents to discuss.
+
+After completing the mini-atlases, SKK, in coordination with the city government and Musrenbang facilitators in each neighbourhood post large copies of the mini-atlases in neighbourhood public spaces. Residents can also print-off this information themselves and use the maps whenever they like.
+
+Data Collection and Analysis Methodology
+Solo city government had never collected the information included in the mini-atlases, let alone aggregated it through GIS and disseminating it more widely, but the administrative organisation of all cities across Indonesia, including Solo, facilitates the collection of this data.
+
+Each neighbourhood in Indonesian cities is formed of administrative units called RTs (roughly equivalent to a US Census Block). A “block captain” keeps track of how many people live on the block, how many poor families, etc. SKK put together a team of community facilitators that liaised with the 2,700 “block captains” in Solo to collect the information to populate the mini atlases. As such, each mini-atlas provides a highly detailed profile of each neighbourhood.
+
+### Resources
+
+*Data collection*: 6 months in total.
+
+*Working with data*: GIS mapping and analysis, design of the mini-atlases, website with searchable information from the atlases. (local GIS consultants and a California-based firm to produce SKK website) - 6 months total over a six-month period.
+
+Note that following the data collection, SKK found differences in the collected data compared to official statistics e.g. the official population estimate is lower than SKK’s count. Rather than SKK having over-estimated the population, it seems more likely that some people were not counted in the official census, perhaps due to dissemination problems, illiteracy, or inadequate follow-up procedures.  While such data discrepancies exist, they are generally not critical.
+
+### Scalability
+
+The experience with the data collection emphasises that this methodology is replicable. As discussed in ‘Follow up projects’ SKK will attempt to replicate the initiative in Solo and Makassar, adding an innovative SMS data collection approach.
+
+### On the Ground Implementation: Using the Data
+
+SKK trained all Solo Musrenbang facilitators in the use of the mini-atlas during the 2010-2011 Musrenbang process. Further, SKK staff collaborated closely with Musrenbang facilitators in 10 neighbourhoods to encourage and observe the utilization of the tool during that Musrenbang cycle.
+
+During the 2011-2012 Musrenbang cycle, SKK staff conducted refresher training for Musrenbang facilitators. Unlike the previous rounds, in four of the 10 neighbourhoods, SKK staff worked with neighbourhood leaders and facilitators to develop a second-generation mini atlas. The latter, known as MA+, sought to amend/ augment the original neighbourhood mini atlas with information that community members noted would enhance the effectiveness of the tool. For example, in one neighbourhood, feedback included adding information about small businesses within the mini atlas.
+
+### Follow-up Projects
+
+SKK has begun a Musrenbang budget analysis initiative with the aim of increasing transparency and developing new planning tools related to the outputs of the Musrenbang process in Solo. To achieve this aim, SKK is reviewing and comparing the lists of proposals made at the RT levels to the respective final city budgets, to determine which projects proposed through Musrenbang were selected and implemented. Ultimately, this information could be utilised to develop maps that visually highlight what type of projects were proposed and where, as well as which were implemented in previous years.
+
+Additionally, with funding from the Ford Foundation, SKK will begin collecting RT-level information from each neighbourhood utilising an innovative SMS data-capturing methodology. Specifically, instead of community facilitators and “block captains” filling out paper forms to aggregate data, SKK will pilot an approach by which the data collection can be conducted using an SMS based cell-phone application. In addition to conducting this work in Solo, SKK will also  implement the project in the city of Makassar.
+
+### Successes
+
+1. In some neighbourhoods the mini atlases served as a catalyst for communities to enhance their discussion of issues and prioritisation of Musrenbang funding requests.
+2. The Solo government officially mandated and included future funding for the printing and use of the mini atlases as part of future Musrenbang cycles.
+3. As part of a city-sponsored design competition, ‘Solo Eco Cultural City’, to create solutions for a sustainable city, student participants downloaded and used the mini atlases to do their planning.
+4. Other city and NGO initiatives are utilising information from the mini atlas to help inform their work. For example, a current City-NGO initiative which will conduct a poverty assessment in a number of Solo neighbourhoods is utilising the mini atlases as an information baseline.
+5. A critical characteristic of the mini atlases was providing clean, universally recognised graphics that could be understood by community members.
+6. The mini-atlas is a tool with relevance beyond Musrenbang. Musrenbang is part of a continuum of planning activities in neighbourhoods that includes RPJMK planning, which is medium-term poverty alleviation planning; PNPM, a World Bank upgrading program, and others.
+
+### Challenges
+
+1. The willingness and ability of neighbourhood facilitators to effectively utilise the mini-atlases has a significant impact on whether they are used or serve their purpose. Some neighbourhoods resisted the use of new tools. In other neighbourhoods, facilitators were not as engaged as they should be with the Musrenbang process (for example, some do not show up to community meetings as they should). SKK is currently working on a set of recommendations to tackle this.
+2. Feedback during RT and RW (groups of RTs) meetings suggests that in many communities the data presented in the mini atlases is too high level. In other words, the mini atlases do not provide sufficient nuance regarding the blocks since they are presented at the neighbourhood level. This issue was a key catalyst in SKKs 2011-2012 effort to develop a second-generation mini atlas.
+3. Given the 18-month Musrenbang budget cycle and a lack of clarity regarding the projects selected and implemented during previous Musrenbang processes, there is a tendency for communities to recycle project proposals yearly. SKK’s ongoing Musrenbang budget analysis project seeks to shine light on this issue.
+
+### Project Resources
+
+Key Actors: A project director, 2 community facilitators, an urban designer, a GIS expert, a website developer,10 community facilitators to communicate with and collect data from RTs, student volunteers (local universities) as map makers/drafters.
+
+Additionally, it is important to note that the project had the support of the Solo mayor. The government worked with SKK to engage the neighbourhood heads to support data collection.
+
+<p style="text-align: right">Thanks to Hector Salazar Salame and Michael Haggerty (SKK)</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-5">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter6-5.md

@@ -0,0 +1,114 @@
+---
+title: Balance your City's Budget - Citizen Budget, Open North
+---
+
+## Balance your City’s Budget : Citizen Budget, Open North
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="no" title="Publish Better Data" />
+            <img src="../images/educate.png" class="no" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Canada</td>
+    </tr>
+	<tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2">Local government</td>
+	</tr>
+	    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://citizenbudget.com/">http://citizenbudget.com/</a></td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">All levels of government</td>
+    </tr>
+</table>
+
+<img alt="Citizen Budget" src="http://farm8.staticflickr.com/7229/7272472996_6013c841d1_o.png" class="screenshot" />
+
+### Background
+
+Citizen Budget is a project of Open North, a Canadian non-profit that creates websites to increase government transparency and promote citizen participation. Open North sells Citizen Budget to municipalities as licensed software or as a service.
+
+Citizen Budget is an online budget simulator that focuses on a local government's controllable operating expenses. The tool challenges citizens to re-balance the budget by increasing or decreasing the amounts allocated to municipal activities, local services and revenue sources. The municipality defines the list of activities and services and their associated amounts.
+
+Once the online consultation is over, Open North prepares a report for policymakers at the municipality,  to allow them to better incorporate citizens' priorities in their budgetary decisions. This report evaluates how representative the participants are of the population, analyzes the data, and identifies trends, which it reinforces with data visualizations.
+The goal is to consult citizens on their budget priorities, while educating them about the local budget and raising awareness of the difficult choices and trade-offs in balancing a budget. Other potential customers include police and fire authorities, school districts and labor unions.
+
+Citizen Budget's first client was the borough of Plateau Mont-Royal in the City of Montreal, regarding their 2012 operating budget. Open North is now looking for municipalities who would like to consult citizens on their 2013 operating budget. This project can be tailored to allow citizens to submit input on printed forms or using their mobile devices.
+
+### How it works
+
+Unlike many of the applications in the census, which asked users to vote on abstract priorities e.g. increase health spending by X million (voting in amounts which, without specialist knowledge, do not mean an awful lot to the average citizen who (generally relate more easily to e.g. the scale of their household budget), citizen budget allows users to vote on concrete and tangible outcomes, which are aligned with decisions that councillors might have to make themselves (build more of these, agree to close libraries on Sundays to save money.) Citizens have to produce a balanced budget, so are forced to trade-off priorities against one-another. The voting system is incredibly simple and allows the user to understand the trade-offs and impacts of their given choice in context.
+
+### How long did the project take to implement?
+
+The project is constantly improving and so work is ongoing. The current iteration of the project took 10 days for the technical implementation, with another 10 days for data collection, meetings with government, etc.
+
+### Successes
+
+Online applications can reach a larger and more diverse group of citizens and can offer more personalized information than traditional offline methods. Unlike council meetings or face-to-face consultations, citizens can participate online anytime, from anywhere, using their favourite Internet device.
+
+They can take the time to express themselves carefully without having to wait to be given the floor. In short, online solutions reduce many barriers to participation.
+
+This project has so far been run in one city district, the borough of Plateau Mont-Royal in the City of Montreal. Out of an adult population of 89,000, 3,160 visited the budget simulator and 363 submitted balanced budgets that expressed their budget priorities. The local government used the major trends in residents' budget priorities to inform their budget decisions - principally, to allocate more money to make the borough greener and cleaner.
+
+Residents expressed their appreciation for the consultation and gained a greater appreciation for the difficult choices and trade-offs that elected officials face when balancing a budget.
+
+Many citizens from outside the borough expressed interest in having a similar initiative in their territory.
+
+### Challenges
+
+1. The project did not have government staff dedicated to the project's promotion, and so did not reach the greatest number of participants.
+2. The government did not effectively communicate to citizens the way in which it used the input from the online consultation.
+3. The major challenge is for the government to determine the actual cost of individual activities and services. Budgets do not tend to be organized per activity, and so a fair bit of work has to go into coming up with these numbers.
+
+#### Do users require any special skills to use the platform?
+The project requires citizens to have a basic level of competency in the following: Internet use, reading skills, budget literacy, and familiarity with municipal activities and local services.
+
+#### What skills are required to implement the project?
+
+The project requires a web developer and a designer to create the application and a statistician/analyst to prepare the report on the consultation for the government. Additional non-technical staff are needed to educate and interact with governments, to manage the other team members, to plan the promotion of the consultation, etc.
+
+### Other comments
+
+1. Consultations cost money. With many local governments facing significant budget constraints, many find it difficult to justify (to themselves and to residents) the cost of a consultation.
+
+2. Politics. Some governments may worry about the consultation being a negative experience, especially if residents are already not showing support. They may worry that residents will support a policy or initiative that goes against the government’s promises, platform or beliefs. Finally, a particular consultation method may be seen as a vestige of a previous government, and therefore something to get rid of.
+
+3. Many local governments would want an offline consultation in addition to any online consultation, e.g. to get input from harder-to-reach demographics. Even if Open North, as a service provider, makes the online consultation as easy-to-run as possible, some local governments will not use it due to the planning, preparation, operating and follow-up costs of an offline, face-to-face consultation.
+
+4. It is difficult to get a local government to run a consultation year after year. Most will perform it as a one-off initiative. It is considered more in terms of its publicity, marketing or communications value, rather than its democratic value.
+
+Such systems have the potential to better inform and educate citizens about decisions and processes. By sharing decisions with citizens, they can become more efficient and effective in the delivery of services. Governments provide much of the information for these initiatives. However, without government buy-in, these initiatives would not have the power to effect change.
+
+<p style="text-align: right">Thanks to James McKinney (Open North)</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-6">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter6-6.md

@@ -0,0 +1,119 @@
+---
+title: Mobile-Enhanced Participatory Budgeting - the World Bank
+---
+
+## Mobile-Enhanced Participatory Budgeting : the World Bank
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="no" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="no" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="no" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="" title="Mobile Technology" />
+            <img src="../images/web.png" class="no" title="Web-based Technology" />
+            <img src="../images/offline.png" class="" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="no" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Democratic Republic of Congo (also been done in Dominican Republic)
+Mobile Voting has also taken place without the Bank in: Ipatinga Brazil, La Plata (Argentina), Belo Horizonte (Brazil)</td>
+    </tr>
+	<tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2">Local government</td>
+	</tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Individual citizens</td>
+    </tr>
+</tr>
+<tr>
+    <th class="inner">Source Document</th>
+    <td colspan="2">This section contains excerpts from <a href="http://bit.ly/ITseQN">Estefan and Weber’s short article</a></td>
+</tr>
+
+</table>
+
+### Background
+
+After years of conflict in the Democratic Republic of Congo, many are unable to access the most basic of public services. While many citizens in the province don’t have access to water or electricity in their homes, they do have mobile phones. The World Bank Institute’s ICT4Gov program has introduced[m] mobile technology to enhance participatory budgeting processes to help local authorities decide on the allocation of available resources according to citizens’ priorities. In a community-chest style process, the local government devotes a percentage of the local investment budget to the project deemed most important by the citizens.
+
+The World Bank played the role of facilitator in a number of spheres for the project - they tailored software, negotiated with cellphone operators and built capacity to demonstrate to local populations how the technology works. In each use-case, the Bank provided the local community with a portfolio of options that technology could offer and allowed the community to decide for themselves which options to deploy. The Bank was also instrumental in other processes such as drafting the text messages which were used to contact constituents and building capacity in understanding the budget. In the case of the Democratic Republic of Congo, the Bank played a role in bringing local governments, civil society, universities etc. together to work out how to implement these projects.
+
+### The Role of Technology
+
+ICT4Gov is using mobile phones for four purposes:
+
+* To invite citizens to participate offline in participatory budgeting assemblies through geo-targeted SMS messages. These messages, reaching all the phones receiving signal from a particular tower, announce the date, time and location of the assemblies.
+
+Besides the blanket approach, the Bank is also experimenting with manually collecting phone numbers (this was done in the Dominican Republic). When collecting numbers, it is also possible to collect basic information about the person in question, such as their gender, and the neighbourhood they live in, so that targeted messages can be sent out (e.g. low attendance rates from women can be counteracted by sending invites just to women.) So far, the Bank have sent around 2000 messages this way compared to 300,000 through the blanket approach.
+
+*  Mobile phones were also used for voting (DRC), allowing the citizens to send a text to identify which of the priorities they would like to see addressed in their community. This is being trialled in a controlled environment at the in-person meetings. This ensures that citizens are present throughout the discussions and so are able to make informed decisions. During meetings, the participants rank their priorities on a scale of 1-4. This makes meetings more efficient as counting is done electronically and feedback can be monitored. In the Dominican Republic participants had the option to vote remotely.
+
+* As an announcement service to communities to broadcast the the result of the vote, making the process more transparent and inclusive.
+
+* For feedback on the projects that were chosen. Through text messages, citizens are able to offer feedback and monitor the projects.
+
+![Perception DR](http://farm8.staticflickr.com/7245/7272472242_28b8506119.jpg)
+<p>
+<small>Voters in the Dominican Republic were asked how they felt about having the option to vote remotely via SMS.</small>
+</p>
+
+### Project Resources
+
+1. Knowledge of participatory design and community operation
+2. Development knowledge in mobile telephony sector
+3. Outreach skills
+4. Cost of SMS: In the DRC one million messages were negotiated at 10,000 USD
+
+### Timeframe
+
+1. Training on PB: 8 cities - 2 months
+2. Development: 2 months
+3. Running a session: 2-4 hours
+
+
+### Scalability
+
+There are 1500 cities around the world where participatory budgeting already takes place, and this approach could be considered as an option in them. Using mobiles could be used to encourage citizens is an approach that could be used in other contexts to promote citizen participation, not just PB.  In some countries, such as Estonia, mobile voting has already been used in municipal elections and could be extended to other contexts.
+
+###Successes
+
+1. When the participants were asked what motivated them to take part in the meeting, the majority of respondents mentioned that the SMS had been the strongest deciding factor.
+2. Since the beginning of the program, as the Provincial government sees an increase in the capacity of local government to allocate resources, communities involved have already seen an increase in transfer of funds from the Provincial to the local level.
+3. The preliminary results of an external evaluation suggest that an increase of tax collection at the local level has been associated with the implementation of participatory budgeting. This is based upon testimonials offered by officials and numbers collected with the municipalities. However, given the high number of potential exogenous factors that could have contributed to this result, further assessment should be done before claiming a causal relationship. For the first time, communities such as Ibanda have gone from not having any investment budget to having 40% of their budget devoted to investments. In 2011, the Ministry of the Budget started the process of institutionalisation of Participatory Budgeting in the Province.
+4. The increase in the transfer of funds from the provincial to the local level has benefited communities, which now have more resources to deliver public services to the poor. For instance, the process has made it possible to begin repairing 54 classrooms and a bridge in Luhindja, to create a health centre and repair the sewage system in Bagira, and to build a water fountain as well as toilets in local markets in Ibanda.
+
+![Find out 1](http://farm9.staticflickr.com/8164/7272472560_e4338cbdd7.jpg)
+![Find out 2](http://farm9.staticflickr.com/8017/7272472430_f6ee268408.jpg)
+<p>
+<small>Participants in the Dominican Republic were asked how they had found out about the meeting. </small>
+</p>
+
+### Challenges
+
+1. Collecting numbers requires a lot of resources (Dominican Republic)
+2. With some bulk message providers, some messages did not deliver until many hours later, some recipients received messages in the middle of the night!
+3. More work needs to be done to help people monitor the execution of the projects. People are engaged in the process and feel empowered when they vote, but there is little follow-up and people could begin to become disillusioned with the project if they do not see results.
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter7-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter6-intro.md

@@ -0,0 +1,25 @@
+---
+title: Chapter 6 - Introduction
+---
+
+## Chapter 6 - Case Studies: Where Does the Money Go?
+
+<img alt="Spending" src="http://farm8.staticflickr.com/7105/7274044350_4596ffd63f_m.jpg" class="inline-image" />
+
+The following case studies all feature organisations who re-use the fiscal data released by governments about where money is spent. They seek to promote better specialist and public understanding of the data and increase participation in the budgeting process.
+
+There are two main sections within this ../chapter:
+
+### Presenting Data to the Public
+
+1. [Grading Government Spending](../chapter6-1): Clearspending, Sunlight Foundation
+2. [The Media and Financial Data](../chapter6-2): how do we get this info into the public eye?
+
+### Deciding where the money goes: Participatory Budgeting
+
+1. [Introduction](../chapter6-3)
+2. [Mini atlases for participatory budgeting](../chapter6-4): Solo Kota Kita, Indonesia
+3. [Citizen Budget](../chapter6-5) - Open North, Canada
+4. [Mobile-Enhanced Participatory Budgeting](../chapter6-6) - the World Bank
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter6-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter7-1.md

@@ -0,0 +1,105 @@
+---
+title: Introductions
+---
+
+## From Fractions to Millions: Challenging corruption using mobile phones
+
+
+
+
+<table class="iconmatrix">
+  <tr class="icons">
+    <th class="inner">Fiscal Scope</th>
+    <th class="inner">Project Aims</th>
+    <th>Technology</th>
+  </tr>
+    <tr class="iconbar">
+    <td class="inner">
+      <img src="../images/revenue.png" class="no" title="Revenue Side" />
+      <img src="../images/spending.png" class="" title="Spending Side" />
+      <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+    </td>
+    <td class="inner">
+      <img src="../images/upload.png" class="" title="Publish Better Data" />
+      <img src="../images/educate.png" class="" title="Educate Citizens" />
+      <img src="../images/citizen.png" class="no" title="Facilitate Direct Participation"/>
+      <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+      <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+    </td>
+    <td>
+      <img src="../images/mobile.png" class="" title="Mobile Technology" />
+      <img src="../images/web.png" class="no" title="Web-based Technology" />
+      <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+      <img src="../images/piechart.png" class="no" title="Data Visualisation and Maps" />
+      <img src="../images/standards.png" class="no" title="Formats and Standards" />
+      <img src="../images/social_media.png" class="no" title="Social Media" />
+      <img src="../images/radio.png" class="no" title="Radio" />
+    </td>
+  </tr>
+  <tr>
+    <th class="inner">Geographical Coverage</th>
+    <td colspan="2">India</td>
+  </tr>
+  <tr>
+    <th class="inner">Scope</th>
+    <td colspan="2">Local government</td>
+  </tr>
+  <tr>
+    <th class="inner">URL</th>
+    <td colspan="2"><a href="http://bit.ly/fractions-to-millions">bit.ly/fractions-to-millions</a> (Vivek Srinivasan's article). All quotes from Vivek Srinivasan’s article.</td>
+  </tr>
+  <tr>
+    <th class="inner">Stage</th>
+    <td colspan="2">Pilot Project</td>
+  </tr>
+  <tr>
+    <th class="inner">Users/Audiences</th>
+    <td colspan="2">Rural poor and beneficiaries of local government programs</td>
+  </tr>
+
+
+</table>
+
+
+
+### Background
+
+In recent years, India has been one of the leaders in FOI legislation, where in many aspects the Right to Information Act was seen as more progressive and powerful than legislation in other [developed countries](http://www.guardian.co.uk/society/2012/apr/10/india-freedom-of-information?newsfeed=true). The possibility of digitising public finance records has lead to a significant decrease in the per unit cost of accessing information. "Mazdoor Kisan Shakti Sangathan (MKSS) and other NGOs involved in India’s right to information movement realised that merely having access to government records was not enough given the sheer complexity of the records and people’s ability to understand them."
+
+Social audits in India typically involve comparing the official records of government projects (e.g., cash books, muster rolls, measurement books, supply lists) and validating whether these projects actually existed in reality. This information verification takes place through door-to-door surveys, where actual records are compared against the locals' testimonies - where discrepancies indicate acts of corruption. "The power of this process is evident by the fact that even though very few convictions happen on the basis of the findings of a social audit, data shows that the levels of corruption have reduced appreciably in places where audits have been organized regularly."
+
+The method of the social audit involves accessing the official record and verifying it with the person who should have received the goods or services. Yet understanding the particularities of the project requires training and expertise, and the actual survey requires considerable time and resources to conduct. Mobile phones could be used as a technological solution to social auditing. "Through basic mobile phone SMS technology, official records on basic individual entitlements such as pensions, subsidized food grain, and maternity entitlements could be delivered to individuals via monthly text messages."
+
+“The project has now received the commitment of officials in the states of Bihar and Andhra Pradesh in India and a team that we helped organize is just starting to build the basic technology to store and disseminate public records. We will start with select programs such as the National Rural Employment Guarantee Act in India and expand the coverage over the next few years.”
+
+### The Role of Technology
+
+The Indian Census showed that around 50% of Indians have mobile phones even in rural communities, whereas access to the internet is limited, which highlights the potential of using mobile technology to reach out to households without Internet access. "The main benefit of using SMS technology is that it requires very limited skill, knowledge, or effort from the user. And, it eliminates costly and time-consuming in-person surveys and audits. By using this technology, official information can be disseminated on a regular basis, unlike in the current model where social audits are done sporadically."
+
+### Scalability:
+
+A planned elaborate survey would compare villages where the technology has been implemented to villages where it has not in order to measure the effect of mobile technology on combating corruption. The existing legal framework and the low cost of making public finance information available mean that more citizens and other organised groups will be able to combat corruption.
+
+“At a later stage of the project, it may be possible to add other features that will enable the victims of corruption to take action using mobile phones. For example, the SMS could include the phone number of responsible officials, or an NGO could collect the complaints and initiate action on their behalf.”
+
+### Successes
+
+1. Using SMS technology will facilitate more widespread and more  frequent social audits, which are instrumental in tackling corruption’
+2. “[it] arms individuals with precise information – something they never had before — that officials cannot argue or ignore.”
+3. The resource could be used by people with limited skills or access to the Internet
+
+### Limitations
+
+"Like any technology tool, this of course has its limitations. One of the critical functions of social auditing in India has been its role in mobilizing the general public. The process of gathering people together face-to-face in a public meeting creates a collective energy, which can motivate people to fight corruption."
+
+### Issues with the Data
+
+Much of the data is in printed format and should be digitised in order to conduct the project.
+
+## Project Resources
+
+Estimated project cost: 400,000 USD (including an elaborate survey to measure corruption)
+
+<p style="text-align: right">Thanks to Vivek Srinivasan</p>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter7-2">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter7-2.md

@@ -0,0 +1,18 @@
+---
+title: Introductions
+---
+
+## What Can Governments Do to Help Social Auditors?
+
+There are considerably fewer projects in this section than in the participatory budgeting section. The reasons for this are numerous, but projects often suffer from lack of timely and granular data from governments and the feedback mechanisms to enable citizens to make their voices heard. Promoting participation in government audits should not be perceived as a threat, but rather an opportunity: auditors and civil servants cannot be everywhere, but citizens can, and can provide feedback on issues as and when they perceive them, allowing the government to hone in on problem areas when they need to. A few suggestions:
+
+1. Collaborate: Support and partner with civil society projects which have already developed auditing tools and use the generated feedback for decision-making and public administration oversight.
+2. Implement Standards: Implement accepted data standards and formats in order to ensure the financial data can be categorised and connected back to specific entities and projects.
+3. Allow Anonymous Feedback: Mechanisms by which citizens can anonymously submit information regarding conflict of interests regarding MPs, allowing for the disentanglement of the politicians’ interests, where personal benefits might influence the direction of public policy and public procurement: e.g websites such as [Inspector de Intereses in Chile](http://www.inspectordeintereses.cl/), encouraging mobile feedback, or employing official hotlines such as those used by Supreme Audit Institutions (SAIs). 
+4. Organise Structured Feedback Mechanisms: Beside providing the framework for auditing tools, governments should ensure that the feedback is assessed and acted upon in a structured, timely and efficient manner. This makes it clear who is responsible for assessment and responding to feedback.
+5. Provide Online Training and Support: Training classes and guidance materials about the audit processes for other governments, public officials, civil society and interested citizens could be also made available online with webinars, shared presentations and open training tools.
+6. Ensure Collaborative and Participatory Process: Establish spaces for cooperation between the SAIs and the civil society organisations as they have mutually complementary roles and resources (Nino 2010).
+7. Crowdsource Problem Areas: Providing mechanisms for citizens to give feedback on problems as they experience them. Once an area of concern is identified e.g a particular department is  frequently flagged up as asking for bribes when issuing licences, the government would know to investigate further. 
+8. Lead a Transparent Process End to End: Clear communication and documentation about how the auditing information is used and acted upon.
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter8-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter7-intro.md

@@ -0,0 +1,16 @@
+---
+title: Chapter 7 - Introduction
+---
+
+## Chapter 7 - Case Studies: The Invisible Money 
+
+<img alt="Invisible Money" src="http://farm8.staticflickr.com/7221/7274044300_5b1f9b09a1_o.gif" class="inline-image" />
+
+Here we examine projects which track institutional corruption and malpractice. Social auditing, in the sense used here, is when non-governmental organisations or collections of citizens ‘audit’ government projects. There are a wide variety of organisations who engage in this important work around the world. As well as the examples listed below, the appendix provides further examples of technology which has been used to tackle problems similar to the issues facing organisations conducting social audits (which include violence against auditors, outreach and spreading the message and text-message based reports of service delivery). 
+
+### Social Audit 
+
+1. [From Fractions to Millions](../chapter7-1): Challenging corruption using mobile phones 
+2. [What Can Governments Do to Help Social Auditors?](../chapter7-2)
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter7-1">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter8-intro.md

@@ -0,0 +1,120 @@
+---
+title: Chapter 8 - Putting the Parts Together
+---
+
+## Chapter 8 - Putting the Parts Together - Getting off-budget on-budget OpenSpending & Publish What You Fund
+
+<table class="iconmatrix">
+    <tr class="icons">
+        <th class="inner">Fiscal Scope</th>
+        <th class="inner">Project Aims</th>
+        <th>Technology</th>
+    </tr>
+    <tr class="iconbar">
+        <td class="inner">
+            <img src="../images/revenue.png" class="" title="Revenue Side" />
+            <img src="../images/spending.png" class="" title="Spending Side" />
+            <img src="../images/invisible_money.png" class="" title="Off-Budget" />
+        </td>
+        <td class="inner">
+            <img src="../images/upload.png" class="" title="Publish Better Data" />
+            <img src="../images/educate.png" class="" title="Educate Citizens" />
+            <img src="../images/citizen.png" class="" title="Facilitate Direct Participation"/>
+            <img src="../images/decision-maker.png" class="" title="Get Feedback to Policy Makers" />
+            <img src="../images/data_analysis.png" class="" title="Analyse and Understand Data" />
+        </td>
+        <td>
+            <img src="../images/mobile.png" class="no" title="Mobile Technology" />
+            <img src="../images/web.png" class="" title="Web-based Technology" />
+            <img src="../images/offline.png" class="no" title="Offline and Print on Demand" />
+            <img src="../images/piechart.png" class="" title="Data Visualisation and Maps" />
+            <img src="../images/standards.png" class="" title="Formats and Standards" />
+            <img src="../images/social_media.png" class="no" title="Social Media" />
+            <img src="../images/radio.png" class="no" title="Radio" />
+        </td>
+    </tr>
+    <tr>
+        <th class="inner">Geographical Coverage</th>
+        <td colspan="2">Uganda (could be re-used in other countries)</td>
+    </tr>
+    <tr>
+	        <th class="inner">Scope</th>
+	        <td colspan="2"> central government and international</td>
+	</tr>
+    <tr>
+        <th class="inner">URL</th>
+        <td colspan="2"><a href="http://openspending.org">http://openspending.org</a> (OpenSpending site)  <a href="http://bitly.com/uganda-example">http://bitly.com/uganda-example</a>
+	</td>
+    </tr>
+    <tr>
+        <th class="inner">Users/Audiences</th>
+        <td colspan="2">Donors, recipient country governments, citizens, NGOs. </td>
+    </tr>
+</table>
+
+<img alt="OS + PWYF" src="http://farm8.staticflickr.com/7092/7272477056_b1cf5eafd2_o.jpg" class="screenshot" />
+
+### Background
+Aid flows often do not pass through a recipient government’s conventional budget mechanisms. When this happens, recipient governments themselves may not have the complete overview of where aid money goes and how donor priorities align with their own. This information is vital for governments and aid donors to be able to make the best use of scarce resources.
+
+Normally this overview is not available – leading to waste, overlap and inefficiency. The lack of comparable information means aid donors and recipient country governments can’t work together to coordinate their efforts; it decreases developing country governments’ ownership and undermines the potential for good governance and planning. Donors and governments need to know what others are doing - and crucially, what others are planning on doing - if they are to make sure that these resources are used most effectively. Otherwise, some sectors and areas will not receive enough funding, while others may have too many donors involved.
+
+This project was an effort to combine two key types of fiscal data - revenues from aid and spending information - and present them together in an informative way through an interactive visualisation.
+
+#### How long did the various stages of your project take to implement?
+
+Data Collection: The first step was a huge data-collection effort by the Overseas Development Institute (ODI) and Publish What You Fund (PWYF). It took just over 6 months for the initial data collection, cleaning, and report, with a large amount of manual work by many different people. Not only did the ODI have to manually collect all this data on donors’ aid spending, they then had to map it all to Uganda’s budget. For data collection, detailed financial information was provided by the Ministry of Finance and this existing data for each development partner (or donor) was sent to them for verification, correction and completion, in the form of an MS Access database.
+
+Data Wrangling: 6 months. This data was still not machine-readable and capable of being analysed as it was spread across five different tables. There were other problems as well, e.g. no common currency throughout. PWYF processed the data so that it was in a format suitable for importing into the Open Knowledge Foundation’s OpenSpending software.
+
+Development: 1 man month. The Open Knowledge Foundation created the BubbleTree visualisation so that it was possible to see multiple dimensions of the data at the same time (you can see both which sectors the money is going to, and how that is broken down by type of spender - donor, project aid, budget support).
+
+#### What kind of skills/resources were/are required for the projects?
+
+Expert knowledge of budget and aid data. Data wrangling capabilities. Development skills.
+
+### Successes
+
+1. The data collected in this project was far more comprehensive than the data in the Government of Uganda’s budget. In fact, for the Financial Year in which the report was being conducted (2006/7), donors planned to spend almost double the amount of project-based aid compared with what the Government of Uganda was aware of.
+2. The project proved that it was possible to collect all the necessary data to be able to do this type of analysis for an individual country. Standards, such as IATI, make it easier for such approaches to be replicated at scale.
+3. The visualisations drew attention to a couple of interesting patterns e.g. very occasionally aid money showed up in defence spending, big chunks of money going to disaster management and the north. It also made it possible to establish and compare how donors are (or not) aligning to the policy priorities of the Ugandan government.
+
+### Challenges
+
+The project required considerable human resources to clean and collect the data. If data had been published in a consistent and machine readable format, this would have been considerably easier.
+
+The feedback below comes from people we asked to review the platform:
+
+1. When we tried to solicit for feedback and encourage journalists to use the visualisation in their reports, they still asked us ‘where are the stories?’. The visualisations gave a high level overview, useful to assess overall priorities and aid distribution, but more work with capacity building to help journalists understand and work with fiscal data is required to help them find the stories.
+2. Another reason journalists were reluctant to cover the story was that the data was not up-to-date enough. The most recent data available was from 2006 and the visualisation was completed in 2011.
+3. Further work needed on explanatory texts - what exactly can you do with the visualisation and what are the known limits?
+4. Visualisation inevitably implies a level of editorial judgement. We received a comment that if Uganda receives general budget support, all the bubbles should show a sliver of aid (since general budget support funds the overall budget, not specific sectors). In the current version, many sectors do not show a budget support component. The decision to show this at the top level only, and not in each of the sectors, was taken because by that point (i.e. at the sectoral level) it is then considered part of the Government of Uganda’s revenues.
+
+### Scalability
+
+OpenSpending is an international platform which allows anyone to upload and visualise government financial data. The database already holds many international, national and sub-national datasets and the software can easily be translated into other languages. OpenSpending is open source and open to contributions on any level from the community; contributing data or code and translation are the most common activities.
+
+A similar approach could be taken in other countries, and work currently underway on the standardisation of aid-sector codes may make this easier. In addition, possibilities of mapping IATI compliant aid data onto COFOG compliant budgets are being researched, although more granular standards may be required in order to accurately represent the data<sup>[1]</sup>. If standards were widely adopted, this approach could be replicated.
+
+To extend the project further and make it even more useful, the following are needed:
+
+1. Aid information (but crucially, it needs to be timely, detailed and comparable): Donors representing over 30% of global Official Development Assistance already publish their data as of 18/11/2011, but more work will be needed to encourage the other 70% to do so.
+2. Budgets in a machine-readable format: some governments have already begun publishing these (e.g. Kenyan open data initiative); many others have budgets available online in PDF (including Uganda, Sierra Leone, for 2010).
+3. Mapping from aid to budgets: work is currently underway in this area.
+
+### Where next?
+
+Possible further areas for exploration and development of the platform:
+
+1. Seeing budget in perspective of the legislative process, although this is a) difficult and b) only part of the story, as a lot of the most interesting changes happen when the Budget has been agreed and moves to the Executive.
+2. Heuristics: On a basic level, show average, maximum, minimum aid donations to a sector. Variance: flagging differences above/below a certain amount
+3. Comparisons, e.g. Spending relative to other districts. Ability to see the context of your current view: (what filters have you selected?).
+4. Feedback and comment features, ability to annotate data points as well as collections / facets of the data e.g. to show absence of data.
+
+<p style="text-align: right">Thanks for input from Mark Brough</p>
+
+<div class='footnote'>
+    <sup>[1]</sup>COFOG, which was designed specifically to describe the activities government undertakes, is an appropriate starting point to examine alignment between recipient budget classifications and existing international aid classifications. COFOG represents country sector and administrative classifications fairly well at aggregated levels, but at the lower levels tends not to disaggregate the functions of government in the same ways or to the same degree that many governments do‘ (Moon and Mills, 2010).
+</div>
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter9-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/chapter9-intro.md

@@ -0,0 +1,88 @@
+---
+title: Final Observations and Review
+---
+
+
+## Final Observations and Review
+
+
+While this report demonstrates possible ways in which technology can contribute to fiscal transparency it should also be clear that technology is not a magic potion that will automatically resolve all barriers to fiscal transparency. 
+
+
+The authors of this report were tasked with highlighting ‘cutting edge’ technology for fiscal transparency. While we have attempted to do so and to show some of the very best and most sophisticated work in this sphere , we would like to emphasise that simple solutions often yield the best results. 
+
+
+In delivering on fiscal transparency, a combination of online and offline, technical and non-technical approaches will be required. In some cases, technology will be neither necessary nor appropriate: e.g. in participatory budgeting it will be crucial to engage citizens offline if the process is to be effective. In other cases, prioritising ICT over offline methods can lead to exclusion for poor and marginalised groups who have little access to the internet or low literacy levels. 
+
+
+Nevertheless, the examples highlighted here do show that technology can be a very effective complement to existing processes in enhancing fiscal processes and transparency. These tools can work on numerous levels simultaneously, reducing barriers to participation and reaching out to groups who would otherwise not have the chance to participate. 
+
+
+We have already highlighted specific recommendations in individual sections, and here we would like to pull together some of the most important. The report demonstrates that many actors need to be involved  in order to promote fiscal transparency: 
+
+
+Below are numerous recommendations related to the role of all parties including governments, civil society organisations and multi-stakeholder initiatives such as GIFT. 
+
+
+### Governments: 
+
+* Promote the use of simple, machine-readable formats which can significantly increase the usefulness of data 
+* Make sure that all fiscal data is released under a proper open license (i.e. one that conforms to [the Open Definition](http://opendefinition.org/))
+* Support and partner with civil society projects to assess whether existing, open-source tools will serve the purpose desired, before looking to create new ones
+* Organise structured feedback mechanisms and make sure teams within governments have the resources to deal with potentially large volumes of feedback 
+* Crowdsource Problem Areas: Providing mechanisms for citizens to give feedback on problems as they experience them. Once an area of concern is identified e.g a particular department is  frequently flagged up as asking for bribes when issuing licences, the government would know to investigate further.
+* Make sure technology does not become a substitute for face-to-face contact. Many important parts of the PB process happen offline. Technology is a useful complement to offline channels of communication and participation, but is not a substitute.
+* Examine structured data formats for drafting legislation and tabling amendments
+
+### Technical Specialists 
+ 
+* Look into ways to make it easier for governments to make data available from the data management systems they already work with - do they have APIs? can they easily pull out reports from their accounting system (e.g. an ‘export report for press’ option)? 
+
+### Civil Society Organisations & Private Sector
+
+* Re-publish data they have derived and added value to. 
+* Work on capacity building and data literacy: Are all necessary skills being fostered to maintain technical solutions? For example, is an intern relied on to load new data or do all members of staff know the standards and processes required to do it? 
+
+### Foundations & Funders
+
+* Many of the projects highlighted in this report are one-off solutions, tailored to an individual local problem. Funders may like to review existing projects (e.g. those listed in the appendix & in the participatory budgeting software census, and on sites such as the [Civic Commons Marketplace](http://civiccommons.org/apps)), to see whether any existing open-source solutions could be adapted to serve other or more general problems, before initiating another.
+* Support platforms (as well as individual instances of a piece of software) which allow solutions to be scaled, promote inter-organisational data-sharing and communities of practice. 
+* Collect resources for long-term archives of data and documents relating to fiscal activities. 
+
+### GIFT: Recap
+
+First and foremost, we see GIFT as being in a strong position to foster technical standards and best practices regarding fiscal transparency. It should be appreciated that especially the release of data, by its nature, is generative: it is nearly impossible and undesirable to predict what people will want or be able to build when data is made both available and usable. Such outcomes should be specific to each country, addressing its political and technical landscape.
+
+Hence, in this report, we have attempted to highlight case studies which are experimental in their nature. We hope that GIFT will be instrumental in supporting further such experimental projects and promoting discourse between governments and civil society in this area.  
+
+#### Promote Open Data as the Raw Fuel for Technology
+
+* Members of GIFT should themselves ensure - whether they are national governments or international organisations - that they publish their fiscal and other financial information in full and in machine-readable formats.
+
+* Promote publication of key budget information as machine-readable data. Work with organisations such as the International Budget Partnership to include a related criterion in the Open Budget Survey. Set up online webinars / workshops showing how data is used after its publication.
+
+* GIFT should initiate the creation of a light-weight, demand-driven standard for the release of structured expenditure information to enable its comparability between countries. 
+
+* Work closely with the Open Government Partnership to promote release of datasets which are key to contextualising financial data, such as economic indicators or procurement data.
+
+* Promote the release of sub-national data, perhaps via local partners, as well as national level.
+
+#### Technology Outreach and Idea Sharing
+
+* Promote experience sharing between governments through workshops and forums in existing social media platforms. Governments need to share the lessons learned in using technology to promote transparency, accountability and participation across different countries.
+
+* Involve civil society actors who use data and services provided by governments, to review and share ideas about existing and planned initiatives. Such actors could form advisory-panels on particular topics e.g. company data.
+
+* Create a well-maintained index of existing initiatives both an the demand and supply side, including methods of data release, re-use and presentation, discussion or activism enabled by such information. 
+
+#### Technology Review
+
+* Review existing projects and and solutions from the point of view of both governments and civil society actors.[Civic Commons Marketplace](http://civiccommons.org/apps) as well as the [appendix](http://bit.ly/TTAPF-projects), but a larger-scale, community effort will be required to keep this up to date. 
+
+* Based on feedback from the above refine and maintain an online, collaborative catalogue of technical and policy options (perhaps as a wiki) which can guide governments about existing solutions and experiences.
+
+#### Technology Sharing
+
+* Promote the use of promising projects from the Technology Review stage. Liaise with funding bodies to ensure open-source solutions are available for all. 
+
+<div class="pull-right"><a class="btn btn-default btn-mini" href="../chapter10-intro">Next &raquo;</a></div>

examples/openspending/content/resources/gift/index.md

@@ -0,0 +1,27 @@
+---
+title: Global Initiative for Fiscal Transparency Report
+---
+
+![Cover of TTAPF Report](http://farm9.staticflickr.com/8027/7296195082_0ae9bdb2f0_z.jpg)
+
+This report was prepared for the Global Initiative for Fiscal
+Transparency ([GIFT](http://www.fiscaltransparency.net) with generous
+support from the Omidyar Network.  The views represented are those of
+the authors.
+
+<table  align= "center">
+
+<tr>
+<td>
+<img src="images/logo_small.png">
+</td>
+</tr>
+<br>&nbsp;</br>
+<tr>
+<td>
+<img src="images/on.jpg">
+</td>
+</tr>
+</table>
+
+{% include gift_nav.html %}

examples/openspending/content/resources/handbook/.gitignore

@@ -0,0 +1,3 @@
+old/*
+book.zip
+book/*

examples/openspending/content/resources/handbook/NOTES

@@ -0,0 +1,9 @@
+
+
+
+https://github.com/esetera/Objavi/blob/master/templates/templating_template.html
+
+http://okfn.booktype.pro/export/spending-data-handbook/export
+
+
+

examples/openspending/content/resources/handbook/README.md

@@ -0,0 +1,19 @@
+Spending Data Handbook
+======================
+
+
+How to generate and export this book: 
+
+* Go to booki.cc and log in. 
+* Visit your dashboard and press import book. 
+* Select "Booktype book" option and use
+  http://okfn.booktype.pro/spending-data-handbook/ for the URL; then
+import.
+* Go to the book and publish it. 
+* Pick up the output from http://objavi.booki.cc/tmp/?C=M;O=D
+* Replace all updated files, esp. body.html 
+* Open in a Chrome with experimental WebKit features (cf
+  http://bookjs.net/).
+* Print :) 
+
+

examples/openspending/content/resources/handbook/body-header.html

@@ -0,0 +1,11 @@
+<html dir="LTR" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+    <head>
+         <script src="//cdnjs.cloudflare.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
+         <link href="vendor/bookjs/book.css" rel="stylesheet" type="text/css" />
+         <link href="okfn-book.css" rel="stylesheet" type="text/css" />
+         <script src="book-config.js" type="text/javascript"></script>
+         <script src="vendor/bookjs/book.js" type="text/javascript"></script>
+         <script src="vendor/sweet-justice.min.js" type="text/javascript"></script>
+</head>
+        <body id="book" class="sweet-justice">
+

examples/openspending/content/resources/handbook/body.html

@@ -0,0 +1,467 @@
+<html dir="LTR" lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+    <head>
+         <script src="//cdnjs.cloudflare.com/ajax/libs/jquery/1.8.3/jquery.min.js"></script>
+         <link href="vendor/bookjs/book.css" rel="stylesheet" type="text/css" />
+         <link href="okfn-book.css" rel="stylesheet" type="text/css" />
+         <script src="book-config.js" type="text/javascript"></script>
+         <script src="vendor/bookjs/book.js" type="text/javascript"></script>
+         <script src="vendor/sweet-justice.min.js" type="text/javascript"></script>
+</head>
+        <body id="book" class="sweet-justice">
+
+    
+    <div id="section-1" Class="objavi-subsection"><div Class="objavi-subsection-heading">Introduction</div><div Class="objavi-chapter"><strong Class="initial">1.</strong> Introduction</div><div Class="objavi-chapter"><strong Class="initial">2.</strong> Working with others</div><div Class="objavi-chapter"><strong Class="initial">3.</strong> Helping the government help itself</div></div><h1 id="NetDspqLaA_0" class="first-heading"><strong Class="initial">1.</strong> Introduction</h1>
+<p>The Spending Data Handbook is addressed to people and organisations who want to use and understand government <strong>budgets</strong> and <strong>spending</strong> data in their work. These groups may include government oversight and accountability groups, issue oriented advocacy organisations, journalists covering the latest round of budget cuts or austerity measures, or citizens who just want to inform themselves on what their government is spending. The truth is that for many countries, it's never been easier to access the underlying data relating to government budgets and spending and the expectations have never been higher for governments to release this information in a timely manner. But once you've got the information, making sense of it can be daunting.</p>
+<p>Irrespective of which kind of user is approaching this data, they all have one thing in common: they need to manipulate, analyse and interpret data in their work<strong><em>.</em> </strong>Increasingly, we are bombarded with statistics and figures from governments and political parties and it is the job of these organisations to scrutinise and fact-check them, as well as come up with alternative models and solutions. This requires timely, data driven analysis.</p>
+<p>If these groups think that <em>"</em>topic-driven analysis<em>"</em> can drive positive change within governments, there is another group at whom this book is addressed: a strange race of people who self-identify as "civic <em>hackers</em>" and think that technology can be put to work to make government as easy, accessible and effortless for citizens as the internet. While these groups use data processing tools in their day to day lives, their focus is often on enabling others to act on information, rather than having a particular advocacy aim themselves (though there are a few notable exceptions to this rule).</p>
+<div id="note">DON'T PANIC! This book uses a bit of technical vocabulary and terms that may be unfamiliar or used differently in different circles. Anything in <em>italics</em> such as <em>hacker</em> in this paragraph, can be found in the glossary at the back, it may not mean what you think.</div>
+<p>The range of topics tackled by these groups is so diverse that it would be impossible to address all of the questions and issues in one humble book. However, there are some overarching principles and helpful techniques which apply universally to working with government financial data and particularly, how technology might help to do so. This book aims to highlight areas where civic hackers, citizens, Civil Society Organisations (henceforth <em>CSO</em>s), and civil servants working in the field of government transparency could combine forces to achieve common aims. Like the Open Data Handbook (<a title="www.booksprints.net" href="http://opendatahandbook.org/en/">http://opendatahandbook.org/en/</a>), it will be available as a continually evolving, open, educational resource on the internet.</p>
+<div Class="objavi-no-page-break"><h2>What we will cover in this book</h2>
+<ul><li>Collaborating with other organizations to pool resources and strengthen your advocacy effort</li>
+<li>If you're just starting out, what data to look for and what to ask for (nay, demand!) from your government</li>
+<li>The 'Data Pipeline': Tricks and tips for finding, wrangling and systematically processing your data</li>
+<li>Getting ambitious, running a technology project</li>
+<li>Presenting your findings to engage the public, media and government</li>
+<li>Lists and appendices of technical and non-technical resources</li>
+</ul></div><div id="imagewrapper"><img src="static/pipeline_1.jpg" alt=""><p><em>The Data Processing Pipeline</em></p>
+</div>
+<div Class="objavi-no-page-break"><h2>How the book was created</h2>
+<p>The book was started at a four day book sprint bringing together organisations from around the world from both a technical and a CSO background. Representatives from the Open Knowledge Foundation (UK), Fundar (Mexico), the <span>Centre for Budget and Governance Accountability (India), the Public Interest Advocacy Centre (Bosnia and Herzegovina), and the Sunlight Foundation (US) contributed to the original document. The Book Sprint was facilitated by Adam Hyde, founder of the Book Sprint method and <a title="www.booksprints.net" href="http://www.booksprints.net">www.booksprints.net</a>.</span></p>
+</div><div Class="objavi-no-page-break"><h2>Why are we writing this?</h2>
+<p>As people who work with this data, we know better than anyone that a few years of a handful groups spreading awareness of spending data to the general public has not been sufficient for enacting earth-shattering change in the budgetary policies and processes in governments around the world. While it's true there's more spending data available than ever before, these efforts have proven to be superficial in some cases and the budget processes themselves have not become much more transparent or participatory.</p>
+</div><p>Conversely, the armchair auditors that were expected to emerge from the citizenry never really materialised. The notion of citizen auditors and engaging with citizens on an individual and collective basis still struggles to make headway. Clearly, we need to take solutions to this problem to the next level. More groups need to be more involved than ever before.</p>
+<div Class="objavi-no-page-break"><h2>What makes this work so tricky?</h2>
+<ul><li>The data people need is not available to the public (and to CSOs).</li>
+<li>Alternatively, the data is not available until after all the important decisions within government have been made.</li>
+<li>It's difficult to simplify data as complex as budgets and spending and make it accessible to a variety of audiences.</li>
+<li>The discourse on budgets and governance in the country is usually replete with jargon and technicalities.</li>
+<li>Even when compelling research findings are presented, there's no <em>sense of urgency</em> on the part of key policy actors. Advocacy with a range of stakeholders becomes important.</li>
+<li>There is duplication of efforts. One CSO may invest two weeks of its resources into painstakingly cleaning up and extracting data from a policy document published as a PDF while another will be doing exactly the same thing.</li>
+<li>Organizations doing this work often lack any kind of peer review process among similar groups.</li>
+<li>There may be skills gaps at many CSOs. Skill and knowledge sharing can help all of them achieve common goals.</li>
+</ul></div><div Class="objavi-no-page-break"><h2>Contribute to the book</h2>
+<p>This book is released under a Creative Commons attribution licence, meaning that anyone is free to use and reuse the material provided that it is attributed to the Spending Data Handbook. You can help by doing one of the following:</p>
+</div><ul><li><strong>Translate it and customise it for your region.</strong> The examples which we include here are the ones which we know best, however, you may feel that in your area, there are topics or examples which would be more relevant. Take the book, remix it and add your own examples.</li>
+<li><strong>Correct it and update it </strong>- treat it like a Wikipedia article, a living document. The only way that this book will stay relevant and factually correct is with the help of you and other people who know your stuff.</li>
+<li><strong>Be inspired </strong>- we are filling the book with lots of examples of visualisation and data so inspire you and people you work with.</li>
+</ul><h1 id="NetDspqLaA_2"><strong Class="initial">2.</strong> Working with others</h1>
+<p>We've outlined some ambitious goals for this book, but you don't have to go it alone! In this section we focus on how to work with other organizations to create powerful advocacy efforts, all driven by hard fiscal data! Every CSO, not just government transparency organizations, can leverage the information in this book. Government spending data touches issues covered by a variety of organizations and journalistic outlets, such as human rights, the environment, poverty, scientific research, education, economic policy, and more. But not all these groups may realize it's importance.</p>
+<div Class="objavi-no-page-break"><h2>Why work together?</h2>
+<p>A common observation has been that CSOs, journalists, and other groups that work with a lot of data encounter a set of similar problems. The problems typically include lack of technical knowledge, duplication of work, lack of wide dissemination of analysis and results, and creating compelling visual products. A lot of these challenges could be mollified if proper channels of communication existed between these organizations.</p>
+</div><p>One of the most important ways to reduce these technical barriers is to maintain an active dialogue and a process of sharing the methodologies and underlying data in analysis you produce. You can avoid the mistakes of your predecessors by reading their methodologies and learning about idiosyncrasies in the data. Or you may be able to point out mistakes another organization has made. Similarly, if you document how your data has been processed and analyzed, other organizations can benefit (more about that in a few chapters).</p>
+<p>Creating communication can be as easy as joining or maintaining a mailing list of groups that work on similar topics or leverage similar data in their work products. Most webmail services offer free group or mailing list functions, such as Google Groups or Yahoo Groups. If you maintain one, be responsive to messages and active in recruiting members. Email is an extremely low barrier for starting a conversation between organizations.</p>
+<p>If you've already got data products that you want to share, try disseminating them in a couple of different ways. In addition to the methods outlined above, try preparing primers or manuals for other groups that want to start working with datasets that you may be particularly experienced with. You can also organize workshops, meetups, and webinars to familiarize groups with the data you work with. If grassroots issue groups can internalize your data products and present them in a contextually relevant way to their user base, then both groups have gained something from the partnership.</p>
+<p>Another good way to facilitate knowledge sharing and communication between organizations is by conducting internship programmes for interested individuals and organisations. However it is important to keep in mind the target groups. It's great to interface with similar spending data focused groups working at different levels of government than your own, but identifying groups that work on completely different issue areas is important. These groups may feel incapacitated to enact change in the fiscal policy as it relates to their own issues. It then becomes an important job for spending data groups to identify and offer their support. It's especially important to establish strong links with groups that serve those minority populations who are often marginalized in government spending priorities.</p>
+<p>The kind of collaboration discussed above can result in a much broader and stronger coalition of organizations that can advocate for fiscal transparency and detailed disclosure of spending data at all levels of government.</p><h1 id="NetDspqLaA_3"><strong Class="initial">3.</strong> Helping the government help itself</h1>
+<p>We've discussed that it's important for CSOs, journalists, and other groups to work together and it's likely that you won't bump up against too much resistance to this idea. But what about when you get pushback from the government even after you've built a strong coalition to advocate for better access to data? Well, one answer you can give them is "Government darling, you're only hurting yourself".</p>
+<p>Whenever the word 'transparency' is mentioned, the first thought that usually springs to mind is a bright light being shone into dark corners of government offices, trying to expose the dark secrets of corrupt bureaucrats and inefficient expenditure. What is often overlooked is that governments also often stand to benefit from more transparent publishing practices. Not only does proactive disclosure help save on the time and cost associated with increased Freedom of Information requests, but other governments within the same nation can benefit from each other as well.</p>
+<div Class="objavi-no-page-break"><h2>Connecting different levels of government</h2>
+<p>Money at the federal level is often transferred to local governments in the form of grants, direct payments or formula payments. When sub-national governments have better, more timely information on national budgeting priorities, it allows them to adjust their own budgets to account for cuts or increases in certain programs. Similarly, many local governments have small staffs and rely on revenue estimates and models at the federal level to estimate their own revenue.</p>
+</div><p>This underscores the need for a consistent, standardized information sharing across governments, as it would allow them to share models, best practices, and software with each other, instead of custom building everything from scratch. Furthermore, it would allow for cross-checks on both the federal side and local side for data auditing purposes.</p>
+<p>The next section highlights a couple of case studies for CSOs to use when trying to convince their government to publish better data.</p>
+<div Class="objavi-no-page-break"><h2>Case study</h2>
+<p><em>Transparency to combat lag-times for Governments in British Columbia</em></p>
+</div><p>When the province of British Columbia built a data portal (<a href="http://www.data.gov.bc.ca/">http://www.data.gov.bc.ca/</a>), its motivations were primarily:</p>
+<ul><li><strong>citizen engagement</strong> - they wanted citizens to better understand the workings of government</li>
+<li><strong>innovation</strong> - they wanted people to build <span>applications and tools using the data</span></li>
+<li><span><strong>making handovers effective</strong> - a large number of the workforce were approaching retirement age, and those in charge wanted to make sure that they handed over the necessary information well in advance</span></li>
+</ul><p>Probably one of the less-expected impacts was seeing how civil servants themselves used the portal. In 2012 approximately one third of all the traffic originated from government computers. The technology enabled faster access to relevant data within the government departments, contributing to better collaboration on policies that required fiscal data. There was also about 20% increase in the number of Freedom of Information (FoI) requests, showing that releasing a small amount of data fuelled wider interest in data.</p>
+<div Class="objavi-no-page-break"><h3>The importance of constant access</h3>
+<p>It is somewhat terrifying to think that a lot of budgeting decisions could made almost entirely in the dark. As soon as one executive budget proposal is finalised and published, work often begins on producing the next one. Within governments, those who have to draw up the next year's plan need access (and quickly) to information, such as actual quarterly expenditures, in order to work out whether a government department is properly resourced from the outset, or is drastically under or over spending.</p>
+</div><div Class="objavi-no-page-break"><h2>Case study</h2>
+<p><em>International Aid Transparency Initiative (IATI)</em></p>
+</div><p>Building on around 20 years of previous work, the first version of the International Aid Transparency Initiative (IATI) standard was launched in early 2011 and aid donors around the world started publishing to it. The IATI specification documents data about both aid donors and aid activities, enabling comparison and encouraging good practice in data management. The IATI standard also includes space to attach relevant documents and details of project results, to allow the standard to build context around the data and become an end-to-end solution, tracking projects from inception to execution.</p>
+<p>The IATI standard has seen far greater uptake than any previous effort in terms of organisations electing to publish their data in a standard format. These successes are often attributed to the multi-stakeholder nature of the design process, including both policy makers and technical experts. From 2009 to 2010, IATI consulted with a wide range of stakeholders on the design of the technical standard, alongside a parallel process to secure donor support for publishing their aid information.</p>
+<p>In the past it was common to respond to information shortages by building a new database. But by working with open data principles, IATI allows a more distributed solution - where information can flow between organisations in many different ways, not just into a central database.</p>
+<div Class="objavi-no-page-break"><h3>Main user groups</h3>
+<ol><li><strong>Parliamentarians</strong> in developing countries gain a better oversight of the aid resources available. Knowing where to allocate resources in their own budgeting processes is vital to ensure that money is spent in the best way. Sometimes, there is transparency-asymmetry between different parts of government. For example, a treasury may be very willing to open up the information it holds, but departments which benefit strongly from aid donations (e.g. departments of health) may be more reluctant to be transparent about aid revenues, as they will not want to 'lose out' from central government budgeting.</li>
+<li><strong>Donors</strong>: Know where their money is going and whether it is being spent correctly.</li>
+<li><strong>CSOs</strong> and <strong>private companies</strong> who monitor aid effectiveness</li>
+</ol></div><p>The benefits of this approach have already been demonstrated, with many CSOs and charities choosing to follow the IATI Standard, although it was developed primarily with governments in mind.</p>
+<p>Donors publish aid information as a feed which can be read by many different applications, including those created by other donors, by the open data community, and - importantly - by software providers that are developing country financial systems. By providing aid information in a standard format, many different users can access the data in the way they need to - and developing countries can see the resources, which are supposed to be flowing to them.</p>
+<div Class="objavi-no-page-break"><h3>Is there anything like this for spending and budgets?</h3>
+<p>At present, no. The way many governments report their financial data is a function of how their budget process works or how they use their accounting software. This may not even be consistent within a government, but vary by department and sub-department.</p>
+</div><p>Could it be useful to create one? We suspect so. The IATI standard for the first time is enabling people to track the money across country boundaries, something which is almost impossible with other types of expenditure, but crucial for those whose job is oversight of government activities and accountability bodies, such as those who work to prevent international money laundering and corruption.</p><div id="section-2" Class="objavi-subsection"><div Class="objavi-subsection-heading">Data Literacy</div><div Class="objavi-chapter"><strong Class="initial">4.</strong> Data-driven advocacy and research</div><div Class="objavi-chapter"><strong Class="initial">5.</strong> Types of data</div><div Class="objavi-chapter"><strong Class="initial">6.</strong> Getting and cleaning data</div><div Class="objavi-chapter"><strong Class="initial">7.</strong> Analysis</div><div Class="objavi-chapter"><strong Class="initial">8.</strong> Using technology in your work</div></div><h1 id="NetDspqLaA_4"><strong Class="initial">4.</strong> Data-driven advocacy and research</h1>
+<p>We are now in a phase where many governments around the world are proactively publishing documents about what they plan to spend (budgets) and actually spend (spending data). Increasingly, this material is available on the internet, so that anybody can access it at any time. Still, too much of the information is released in the form of '<em>documents</em>' rather than '<em>data</em>'. Ideally we need both so that inforamtion can be analyzed, re-used and understood. This chapter is a quick overview of some of the raw inputs required for data-driven advocacy and how it works in practice. </p>
+<div Class="objavi-no-page-break"><h2>What do we mean by machine-readable data?</h2>
+<p>When we speak about data, what we usually refer to is the notion of machine-readable (<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://en.wikipedia.org/wiki/Machine-readable_data">http://en.wikipedia.org/wiki/Machine-readable_data</a>) data. Many of the formats most commonly used for policy papers and long-form reports published by most policy-making institutions are PDF files, Word documents, web pages or closed interactive infographics - do not structure information in a way that lends itself to automated analysis and extraction.</p>
+</div><p>Such documents are formatted for humans (or printers) to interpret, and it can be hard (and in many cases nearly impossible) for a machine to re-construct the elements in the presentation.</p>
+<p>Other formats, such as Excel and CSV files contain a higher level of structured information. For example, in an Excel file you can mark a number of cells and easily calculate their sum. Even more exotic and useful file formats, such as <em>XML documents</em>, <em>JSON API</em>s or <em>Shapefiles</em> may not have easy-to-use viewer applications. You can think of them as the glue that connects different systems on the web, so that different databases can work together in a seamless fashion.</p>
+<div Class="objavi-no-page-break"><h3>Why do CSOs need it?</h3>
+<p>What asking for machine-readable bulk data means for CSOs is simple: you won't have to spend a lot of time manually extracting data from reports into spreadsheets to be able to filter, sort and analyse it - a process which is both time-consuming and can introduce errors.</p>
+</div><div Class="objavi-no-page-break"><h2>What to ask for when asking for data: a checklist</h2>
+<p>In the next section 'Getting Data' - we will deal with asking governments for data (or getting it via other means). To set the scene for this and to work out whether your government actually publishes usable data already, have a quick look at the following questions:</p>
+</div><ul><li><strong>Is the government's data published in a machine-readable format?</strong> E.g. CSV, XML, JSON. While there is nothing wrong with publishing a PDF to support a data release (in fact it is often nice to have a nicely-laid out document to cross reference and sanity-check data) it shouldn't be the only thing published and if you are asking for a policy document, ask for the underlying data in a spreadsheet so you can check the numbers.</li>
+<li><strong>Does the government publish a '<em>data dictionary</em>' to explain the terms used in the dataset?</strong> This should include definitions of column headers, explanations of terms and ranges used within the main body of the data, explanations of any changes in terminology which have been introduced since last time the dataset was released</li>
+<li><strong>How is the data that is being published <em>actually</em> used internally by governments?</strong> Do some sanity checks on the minimum and maximum values of different columns to make sure they fall into the documented ranges and don't seem out of place. Do you see negative values when you don't think you should? Negative values usually mean money owed.</li>
+<li><strong>Is the structure of the data the same across years? If not is there a description of how it changes?</strong> It never hurts to contact the publisher and ask questions about the change and why it occurred. The publisher may have their name and contact details on the report or webpage. If there is no named contact then call the department's enquires number or send a message to their email address asking to meet or discuss the data.</li>
+<li><strong>How aggregated is the data?</strong> What is the number of real-world financial transactions that are expressed by a single line of the dataset you have? For budgets this will mostly be hard to tell - but with transactional expenditure you want to make sure that the data is fairly disaggregated. Ideally, each entry represents a transaction - but even if this isn't true you'll still want to ensure the number is not in the tens or hundreds of thousands (e.g. government programmes as a whole).</li>
+<li><strong>Ask for reference data.</strong> If your budget or spending data is augmented with reference data, make sure you have access to it. This might include functional or category codes on budget line items, location codes for describing recipient location, or codes that indicate the status of the record. </li>
+<li><strong>Ask also for the guidelines people were given when creating the dataset.</strong> This will make it easier to understand what is included within the data, e.g. are the numbers in thousands / millions. </li>
+<li><strong>Final tip: if the data you want is not given then narrow your scope.</strong> Your chances of success will be higher if you narrow the scope of the data you're requesting from the government and you are specific. Government is the de facto keeper of all kinds of data, so parameters that narrow your request are always helpful.</li>
+</ul><div Class="objavi-no-page-break"><h2>An introduction to data-driven advocacy</h2>
+<p class="p1">Is going out and provoking a riot the best way to get a Government to take onboard your message? There are alternatives: hit them with the data hammer instead!</p>
+</div><p class="p1">Making evidence-based policy proposals consists from three major phases: formulating your assumption, analysis (which often leads to re-formulating your assumption, and presenting your data in an engaging way in a policy proposal.</p>
+<div Class="objavi-no-page-break"><h3 class="p1"><strong>Analysing assumptions</strong></h3>
+<p class="p1">Asking the right question is key to getting the most out of your data. We all make assumptions, and our organisation may have a particular standpoint on a given issue. Our first task is always to formulate our assumptions and then interrogate them ferociously. Although we try to be rational in this process, our judgement is often influenced by our subjective goals, values, and beliefs. Sometimes, you'll need to revisit your assumptions several times over to ensure they are valid and you can back them up with data. Once you know your policy problem is definitely a problem, you can work to package it in a way that's appropriate for your target audience. </p>
+</div><div Class="objavi-no-page-break"><h3 class="p1">What is public interest? </h3>
+<p class="p1">Often our job is to act in the public interest by analysing conflicting assumptions and working out which one is more valid. For example, in Greece, Spain, and many other European countries people protest almost everyday as the Government cuts spending to bring down its budget deficit. If the Goverment wanted to keep its current level of spending, but increased taxes to increase its revenue, different citizens groups would still protest depending on which taxes are to be increased. In any case, there will always be more than one interpretation of any Government policy, and interested side to support it, or not.</p>
+</div><div Class="objavi-no-page-break"><h3 class="p1"><strong>Policy analysis</strong></h3>
+<p class="p1">Once we have a well defined policy problem, specific goals, or results different stakeholders are trying to achieve, and corresponding instruments they are using in this process, we may systematically search for the specific data needed to create our own policy proposals. This data can be obtained either from the Government, some other sources e.g. academic journals, private companies, or generated by ourselves. When data is gathered we will use a specific methodology to analyze it, and based on this analysis we will approve or reject our assumptions. If the assumption is rejected, based on our findings we will have to make the new assumption, and start the process from the beginning. If our assumption is approved, we will use our results to make a policy proposal to the Government.</p>
+</div><div Class="objavi-no-page-break"><h3 class="p1"><strong>Policy proposals</strong></h3>
+<p class="p1">For CSOs it is important to recognize who is a decision maker, hence, who you should be targeting with your policy proposal. Policy proposals should be methodologically well structured, evidence-based, open for debate, and scientificaly evaluated. Governments will seldom take our policy proposals as their own policymaking, but may actually change its course of action, get new insights, views, and understanding of the subject. We may also use policy briefs to approach Government officials, or press releases to get the attention of the public. </p>
+</div><p class="p1"><img src="static/cso-3.jpg" alt=""></p>
+<div Class="objavi-no-page-break"><h2 class="p1"><strong>Case study </strong></h2>
+<p class="p1"><em>Fish subsidies</em></p>
+</div><p class="p1">The influence CSOs have on government policy comes from a wide and varied set of activities. These can range from producing a widely shared dataset or infographic which subtly influences the mood of policy makers, to more targeted CSO advocacy and lobbying on issues they are experts.</p>
+<p class="p1">The Fish Subsidies group (<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://fishsubsidy.org">http://fishsubsidy.org</a>) are a nice example of a CSO engaged in targeted activites. Having collected a comprehensive set of data on Fishing subsidies paid under the European Union’s common fisheries policy and they break this down into payments for every EU member state, and then complemented this with activites of fishing. They have produced a report (<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://is.gd/XYPgq5">http://is.gd/XYPgq5</a>) assessing the environmental and social impacts of the Financial Instrument for Fisheries Guidance between 2000 and 2006. This extensive document fed directly into the EU political decision making process.  </p><h1 id="NetDspqLaA_6"><strong Class="initial">5.</strong> Types of data</h1>
+<p>For the purposes of this manual, we have decided to abstract the meaning of the terms "budget data" and "expenditure data" to fit into a broader context. While these terms may have different meanings on a country by country basis, they are intended to be taken as defined in this section, throughout the rest of the manual. In this section, we look briefly at the two different types of data and what questions can be addressed using them.</p>
+<p><img src="static/pipeline-ask-a-question.jpg" alt=""></p>
+<p><strong>Budget data</strong> is defined as data relating to the broad funding priorities set forth by a government, often highly <em>aggregated</em> or grouped by goals at a particular agency or ministry. For instance, a government may pass a budget which contains elements such as "Allocate $20 million in funding for clean energy grants" or "Allocate $5 billion for space exploration on Mars". These data are often produced by a parliament or legislature, on an annual or semi-annual basis.</p>
+<p><strong>Spending data</strong> is defined as data relating to the specific expenditure of funds from the government. This may take the form of a contract, loan, refundable tax credit, pension fund payments, or payments from other retirement assistance programs and government medical insurance programs. In the context of our previous examples, spending data examples might be a $5,000 grant to Johnson's Wind Farm for providing renewable wind energy, or a contract for $750,000 to Boeing to build Mars rover component parts. Spending data is often transactional in nature, specifying a recipient, amount, and funding agency or ministry. Sometimes, when the payments are to individuals or there are privacy concerns, the data are aggregated by geographic location or fiscal year.</p>
+<p>The fiscal data of some governments may blur the lines of these definitions, but the aim is to separate the <em>political documents</em> from the <em>raw output of government activity</em>. It will always be an ultimate goal to link these two datasets, and to allow the public to see if the funding priorities set by one part of the government are being carried out by another part, but this is often impractical in larger governments since definitions of programs and goals can be "fuzzy" and vary from year to year.</p>
+<p><img src="static/budget-spending-data-3.jpg" alt=""></p>
+<div Class="objavi-no-page-break"><h2>Budget data</h2>
+<p>Using the definitions above, budget data is often comprised of two main portions: <em>revenue</em> and <em>taxation data</em> and <em>planned expenditures</em>. Revenue and spending are two sides of the same coin and thus deserve to be jointly considered when budget data is released by a government. Especially since revenue tends to be aggregated to protect the privacy of individual taxpayers, it makes more sense to view it alongside the budget data. It often appears aggregated by income bracket (for personal taxes) or by industrial classification (for corporate taxes) but does not appear at all in spending data. Therefore, budget data ends up being the only source for determining trends and changes in revenue data.</p>
+</div><p>Somewhat non-intuitively, revenue data itself can include expenditures as well. When a particular entity or economic behaviour would normally be taxed but an exception is written into the law, this is often referred to as a <em>tax expenditure</em>. Tax expenditures are often reported separately from the budget, often in different documents or at a different time. This often stems from the fact that they are released by separate bodies, such as executive agencies or ministries that are responsible for taxation, instead of the legislature<span style="font-size: 12px;"> </span>(<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://internationalbudget.org/wp-content/uploads/Looking-Beyond-the-Budget-2-Tax-Expenditures.pdf">http://internationalbudget.org/wp-content/uploads/Looking-Beyond-the-Budget-2-Tax-Expenditures.pdf</a>).</p>
+<div Class="objavi-no-page-break"><h3>Budgets as datasets</h3>
+<p>A growing number of governments make their budget expenditure data available as <em>machine-readable</em> spreadsheets. This is the preferred method for many users, as it is accessible and requires few software skills to get started. Other countries release longer reports that discuss budget priorities as a narrative. Some countries do something in between where they release reports that contain tables, but that are published in PDF and other formats from which the data is difficult to extract.</p>
+</div><p>On the revenue side, the picture is considerably bleaker, as many governments are still entrenched in the mindset of releasing revenue estimates as large reports that are mostly narrative with little easily extractable data. Tax expenditure reports often suffer from these same problems.</p>
+<p>Still, some areas that relate to government revenue are beginning to be much better documented and databases are beginning to be established. This includes budget support through development aid, for which data is published under the IATI (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://www.aidtransparency.net/">http://www.aidtransparency.net/</a>) and OECD DAC CRS (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://stats.oecd.org/Index.aspx?DatasetCode=CRSNEW">http://stats.oecd.org/Index.aspx?DatasetCode=CRSNEW</a>) schemes. Data about revenues from extractive industries is starting to be covered under the EITI (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://eiti.org/">http://eiti.org/</a>) with the US and various other regions introducing new rules for mandatory and granular disclosure of extractives revenue. Data regarding loans and debt is fairly scattered, with the World Bank providing a positive example (<a title="http://www.publicdebtnet.org/public/Statistics/" href="https://finances.worldbank.org/">https://finances.worldbank.org/</a>), while other major lenders (such as the IMF) only report highly aggregated figures. An overview of related data sources can be found at the Public Debt Management Network (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://www.publicdebtnet.org/public/Statistics/">http://www.publicdebtnet.org/public/Statistics/</a>).</p>
+<div Class="objavi-no-page-break"><h3>Connecting revenues and spending</h3>
+<p>It is highly desirable to be able to determine the flow of money from revenues to spending. For the most part, many taxes go into a general fund and many expenditures come out of that general fund, making this comparison moot. But in some cases, in many countries, there are taxes on certain behaviours that are used to fund specific items.</p>
+</div><p>For example, a car registration fee might be used to fund the construction of roads and highways. This would be an example of a user fee, where the main users of the government service are funding it directly. Or you might have a tax on cigarettes and alcohol that funds healthcare grants. In this case, the tax is being used to offset the added healthcare expense of individuals taking part in at-risk activities. Allowing citizens to view what activities are taxed in order to pay for other expenditures makes it possible to see when a particular activity is being cross-subsidized or heavily funded by non-beneficiaries. It can also allow them to see when funds are being diverted or misused. This may not always be practical at the country level, as federal governments tend to make much larger use of the general fund than other local governments. Typically, local governments are more comprehensive with regards to releasing budget data by fund. Having granular, fund-level data is what makes this kind of comparison and oversight possible.</p>
+<p><img src="static/funding_1.jpg" alt=""></p>
+<div Class="objavi-no-page-break"><h3>What questions can be answered using budget data?</h3>
+<p>Budget expenditure data has an array of different applications, but it's prime role is to communicate to it's user broad trends and priorities in government spending. While it can help to have a prose accompaniment, the data itself promotes a more clear-cut interpretation of proposed government spending over political rhetoric. Additionally, it is much easier to communicate budget priorities by economic sector or category than it is at the spending data level. These data also help citizens and CSOs track government spending year over year, provided that the classification of the budget expenditure data stays relatively consistent.</p>
+</div><div Class="objavi-no-page-break"><h2>Spending data</h2>
+<p>For most purposes, spending data can be interpreted as transactional or near-transactional data. Rather than communicating the broad spending priorities of the government like budget data should, spending data is intended to convey specific recipients, geographic locations of spending, more detailed categorization, or even spending by account number.</p>
+</div><p>Spending data is often created at the executive level, as opposed to legislative, and should be more frequently reported than budget data. It can include many different types of expenditures, such as contracts, grants, loan payments, direct payments for income assistance and maintenance, pension payments, employee salaries and benefits, intergovernmental transfers, insurance payments, and more.</p>
+<p>Some types of spending data - such as contracts and grants - can be connected to related procurement information (such as the tender documents and contracts) to add more context regarding the individual payments and to get a clearer picture of the goods and services covered under these transactions.</p>
+<div Class="objavi-no-page-break"><h3>Opening the checkbook</h3>
+<p>In the past five years, there have been a spate of countries and local governments that have opened up spending data, often referred to as "checkbook level" data. These countries include, but are not limited to, the US (including various state governments), UK, Brazil, India (including some state governments) and many funds of the European Union.</p>
+</div><div Class="objavi-no-page-break"><h3>Disclosure thresholds</h3>
+<p>At least two of these countries have imposed seemingly arbitrary thresholds on the size of transactions that are included. For example, the US and the UK exclude transactions under $25,000 and 25,000 GBP, respectively. Are these thresholds appropriate? That can't be known for sure without more information about how these numbers were arrived at. Principally, having thresholds or exceptions to the reporting of this data depends on the underlying systems that drive disclosure of this data. Are these systems linked directly with the accounting systems already used in the government, easing the burden of disclosure? If so, the threshold for excluding transactions should be very low (setting aside for a moment the cases that require redaction for privacy purposes).</p>
+</div><p>If the systems are mostly divorced, as is the case with the US, then it begs the question, why? The more steps and processes between the internal government accounting systems and the public accounting systems, the higher the chance of error in the data and chance for omission of data. It also undermines the primary goal of public oversight if there are separate systems. However, governments often struggle with IT resources and contracting, which presents a tension between releasing any spending data at all, and release that is consistent with the above principles. If a threshold is necessary, then the amount should be consistent in size and scope with the overall expenditure level for that particular government. It is not appropriate, for example, that the threshold for the US State of Maryland spending reporting is also $25,000, when their annual budget is only a fraction of the federal government's budget.</p>
+<div Class="objavi-no-page-break"><h3>Release early, release often</h3>
+<p>Spending data should be released in a relatively timely fashion, at least a monthly or quarterly basis. The timeliness of this data is what allows users to see if the spending priorities in the budget data are being reflected in the spending data. Also it allows the public and government stakeholders to view the current year's spending on a more detailed level as the next year's budget is being decided.</p>
+</div><p>A good example of such release of spending information can be the Indian experience, especially the <span>Employment Guarantee Programme, one of the major National flagship programmes on providing demand-based employment to the rural working age-group population in India. Its Management Information system (MIS) has become the most effective way of getting information on spending on a monthly basis. The data is updated monthly in an accessible spreadsheet format (Excel) at the sub-national government level. This makes the data transparent and available in the public domain to be equally accessed by all. The village level household database has internal checks for ensuring consistency and conformity to normative processes. It includes separate pages for approximately 250,000 local governments at the village level, 6,465 Blocks, 619 Districts and 34 States &amp; Union Territories. The portal places complete transaction level data in the public domain.</span></p>
+<p><span>However, problems related to maintaining an MIS in every state in a functional form and releasing continuous flow of data have been contentious issues. The major concerns emerge from lack in technical capacity as well as cost related issues. A</span> cumbersome back-end system for supplying the data requires installation of a specific software with prerequisite configurations and technical operators with specific capacities. These requirements have raised costs and put a great demand on technology to ensure a continuous flow of data on the programme, specifically in most interior parts of the country and hence affect timely release of data.</p>
+<div Class="objavi-no-page-break"><h3>What questions can be answered using spending data?</h3>
+<p>Spending data can be used in several different areas: oversight and accountability, strategic resource deployment by local governments and charities, and economic research. However, it is first and foremost a primary right of citizens to view detailed information about how their tax dollars are spent. Tracking who gets the money and how it's used is how citizens can detect preferential treatment to certain recipients that may be illegal, or if certain political districts might be getting more than their fair share.</p>
+</div><p>It can also help local governments and charities respond to areas of social need without duplicating federal spending that is already occurring in a certain district or going to a particular organization. Lastly, businesses can see where the government is making infrastructure improvements and investments and use that criteria when selecting future sites of business locations. These are only a few examples of the potential uses of spending data. It's no coincidence that it has ended up in a variety of commercial and non-commercial software products -- it has a real, economic value as well as an intangible value as a societal good and anti-corruption measure.</p><h1 id="NetDspqLaA_7"><strong Class="initial">6.</strong> Getting and cleaning data</h1>
+<p>As you look into a new, data-driven project, you'll find it easy to imagine all the analysis and comparisons that could be done to prove a particular point or to explore some area of government activity. As you then look into the availability of data on a particular topic, disillusionment will quickly set in: information may be hard to find, unstructured or just not available to the public at all. As you search for data, there are a number of possible techniques you may want to investigate. Some are based on the governments cooperation while others use technology to bridge the gap between how information is shared and what is necessary for a data-driven advocacy project.</p>
+<div Class="objavi-no-page-break"><h2>Getting data</h2>
+<div id="imagewrapper"><img src="static/pipeline-getting-data.jpg" alt=""><em>Data processing pipeline</em></div>
+</div><p>An increasing number of governments have accepted the need for pro-active transparency. When releasing open data, governments commit to making machine-readable information available on the web, in a form that enables anyone to use, re-use and re-distribute the data without legal or technical restrictions. Such releases should happen in a timely manner and include detailed, disaggregated data. Many countries have set up dedicated open data portals which provide easy access to all published datasets and relevant metadata (i.e. information about when a dataset was last updated, who published it and where documentation regarding format and contents can be found). The benefit of using open data when it's available includes the ease of access, but also usually a more authoritative base for any further analysis.</p>
+<p>In some cases, open data is provided through an application programming interface (API), often a web-based method for retrieving, searching or even updating the available information dynamically. APIs provide up-to-date data in a granular and filtered form, removing the need to repeatedly process and update source files.</p>
+<p>A common use case for APIs is relatively time-sensitive information, such as procurement calls and contracts which are released every day. In the UK, BusinessLink (<a title="http://opencorporates.com" href="http://www.contractsfinder.businesslink.gov.uk/data-feed.aspx">http://www.contractsfinder.businesslink.gov.uk/data-feed.aspx</a>) provides a number of data feeds which contain information about procurement notices. Similarly, the USASpending portal provides a set of APIs that can be used to retrieve up-to-date grants information for the US federal government (<a title="http://opencorporates.com" href="http://usaspending.gov/data">http://usaspending.gov/data</a>).</p>
+<div id="note">If you are looking for a list of data catalogs from around the world, DataCatalogs.org is curated by experts in this area from around the world. Search for your country and see what you find.</div>
+<div id="lightimage"><img src="http://farm8.staticflickr.com/7127/7631434818_d8f903252c_z.jpg" alt=""></div>
+<div Class="objavi-no-page-break"><h3>Freedom of information</h3>
+<p>Even before the rise of open data, many countries decided to increase the transparency of their governments by introducing freedom of information (FoI) legislation. Such laws enable every citizen to request documents and other material from parts of the government which do not merit special protection (e.g. due to concerns over privacy, national security or commercial confidentiality).</p>
+</div><p>Journalists, activists and CSOs have long had channels of acquiring information. Sometimes, having a good relationship with a press-officer or a civil servant is good enough and making a formal request for information is unnecessary (your friendly press-officer may even feel slightly offended if you don't ask them nicely first). FoIs generate a lot of paperwork (hence grumpy civil servants), so if you do have the contacts, it may be a good idea to ask nicely first!</p>
+<p>Freedom of Information requests often require some degree of preparation, so that the documents or databases that are requested are clearly identified, you know which department or unit is in charge of it and you can address possible concerns over privacy or commercial confidentiality in your request.</p>
+<p>While freedom of information legislation is in force in many countries, it was often made before the need for structured data became apparent - thus many laws do not allow the citizen to specify a particular format. Many governments choose to release information on paper rather than in a structured, digital form, making the data processing step more painful. Still, the legally binding character of freedom of information requests often makes them an invaluable tool in the process of gaining access to financial data.</p>
+<p>FoI requests may be necessary when you want to get more detail on the projects that government money is funding. Often the transactional spending data released will include only a brief description of the project, if at all. To get more information about it, you might need to submit an FoI request. For instance, if you have the high level payment information for a contract that includes the recipient, location and total amount, but you want to know the details of the contract deliverables, you will probably need to submit an FoI request for the full contract.</p>
+<p>A good example of this process is the Sunlight Foundation's request for information on the Airport Improvement Program in the United States. The program accepts applications from airports around the country for infrastructure improvement grants, such as repaving a runway. Each project is assigned a safety priority rating and is prioritized in a queue. The high level spending information for this program was available in USASpending.gov, but since the priority ratings are specific to this program and not spending data in general, they were not included in that dataset. The Sunlight Foundation submitted a FoI request for the full dataset, including the priority ratings. After that, they were able to determine when airports with low priority projects were getting money, and how often. So the lesson is, if you see some interesting patterns in your high level spending data, don't be afraid to dig deeper and ask for more detailed program information.</p>
+<p>Wanting to submit a request, but not sure where to start, who to address your request to or how to write it? Access Info (<a title="http://opencorporates.com" href="http://www.access-info.org/">http://www.access-info.org/</a>) are an organisation who work to help people obtain the information they require from the public bodies that hold it. They have also produced a toolkit (<a title="http://opencorporates.com" href="http://www.legalleaks.info/toolkit.html">http://www.legalleaks.info/toolkit.html</a>) to using FoIs. It's primarily aimed at Journalists, but most of the tips are equally relevant for CSOs.</p>
+<div Class="objavi-no-page-break"><h3>Data scraping</h3>
+<p>Unlike open data or freedom of information requests, data scraping does not rely on the cooperation of government authorities for acquiring machine-readable documents. Scraping refers to the process of transforming unstructured documents - online database interfaces, PDF files or even printed documents into a more structured form (see section below on Optical Character Recognition for more tips on how to do this). Many tools and techniques enable such refinement by re-interpreting documents generated for humans into a more structured form which can be analysed, aggregated and distributed in new ways.</p>
+</div><p>While there is an increasing number of easy-to-use scraping tools which do not require much technical knowledge, more complex processes - such as the automated scraping of thousands or millions of web sites or the mass interpretation of PDF files require some programming so that the process can be fully automated and produce reliable results.</p>
+<p>In some cases, the only way to gain access to a set of figures is through the digitization of printed material. While scanners and optical character recognition (OCR) software can be used to import such documents, the high cost and low data quality generated through this approach often it an unattractive one.</p>
+<p>A very accessible guide to scraping has been published (<a title="http://opencorporates.com" href="https://leanpub.com/scrapingforjournalists">https://leanpub.com/scrapingforjournalists</a>). Again, it brands itself at journalists (they're getting a little spoiled aren't they?), but scraping is scraping and so a worthwhile read for CSOs.</p>
+<p>Another great source for tutorials, as well as being a tool itself to help with scraping is ScraperWiki. Using Scraperwiki (<a title="http://opencorporates.com" href="https://scraperwiki.com/">https://scraperwiki.com/</a>) has the additional benefit of anything that you scrape being available to others, as most of the scraped data goes into a public data store.</p>
+<p> </p>
+<div Class="objavi-no-page-break"><h3>Getting data out of scanned documents</h3>
+<p>When you deal with scanned documents, the crucial step in the extraction process is to have the computer attempt to recognize any characters - letters, numbers and other signs. Optical character recognition (OCR) software is built to do this, accepting scanned pictures and PDF documents as an input.</p>
+</div><p>There are both commercial software products for OCR (such as ABBYY FineReader, <a title="http://opencorporates.com" href="http://finereader.abbyy.com/">http://finereader.abbyy.com/</a>), and some open-source software packages, such as Google's Tesseract (<a title="http://opencorporates.com" href="http://code.google.com/p/tesseract-ocr/">http://code.google.com/p/tesseract-ocr/</a>). In general, the quality of all automatic recognition is limited, and you should make sure to cross check any numbers coming from scanned material against the printed documents.</p>
+<div Class="objavi-no-page-break"><h3>Keeping the data around</h3>
+<p>As you retrieve data from the government (or other sources), it's easy to just consider the websites it has been released on as a permanent resource. Still, experience has shown that data does go away: whether it is through government re-designing its web sites, new policies that retract transparency rules or simple system failures.</p>
+</div><p>At the same time, downloading complete copies of web sites - a process called <em>mirroring</em> - is a fairly well-established technique that can easily be deployed by civil society organisations. Mirroring involves an automated computer program (for a list see: <a title="http://opencorporates.com" href="http://en.wikipedia.org/wiki/Web_crawler">http://en.wikipedia.org/wiki/Web_crawler</a>) harvesting all the web pages from a specified web page, e.g. a ministry home page. In most cases, it is also possible to find old versions of web sites via the Internet Archive's Wayback machine (<a title="http://opencorporates.com" href="http://archive.org/web/web.php">http://archive.org/web/web.php</a>), a project that aims to create up-to-date copies of all public web sites and archive them forever.</p>
+<div Class="objavi-no-page-break"><h2>Cleaning data</h2>
+<div id="imagewrapper"><img src="static/pipeline-cleaning-data.jpg" alt=""></div>
+</div><p><em>Data Processing Pipeline</em></p>
+<p>As you acquire government spending information, you will notice that such data often has many inconsistencies: program names are used inconsistently, financial amounts will be stated in badly formatted numbers, while some data may not be usable at all due to file corruptions. In short: data always needs to be cleaned and processed. In fact, processing, augmenting and cleaning the data is very likely to be the most time- and labour-intensive aspect of your project.</p>
+<div Class="objavi-no-page-break"><h3>Processing stages for data projects</h3>
+<p>While there are many different types of data, almost all processing can be expressed as a set of incremental stages. The most common stages include data acquisition, extraction, cleaning, transformation, integration, analysis and presentation. Of course, with many smaller projects, not each of these stages may be necessary.</p>
+</div><p>In this process, each stage fulfils a unique purpose</p>
+<ul><li><strong>Acquisition</strong> describes gaining access to data, either through any of the methods mentioned above or by generating fresh data, e.g through a survey or observations.</li>
+<li>In the <strong>extraction</strong> stage, data is converted from whatever input format has been acquired (e.g. XLS files, PDFs or even plain text documents) into a form that can be used for further processing and analysis. This often involves loading data into a database system, such as <em>MySQL </em>or <em>PostgreSQL</em>.</li>
+<li><strong>Cleaning and transforming the data</strong> often involves removing invalid records and translating all the columns to use a sane set of values. You may also combine two different datasets into a single table, remove duplicate entries or apply any number of other normalizations.</li>
+<li><strong>Analysis</strong> of data to answer particular questions we will not describe in detail in the following chapters of this book. We presume that you are already the experts in working with your data and using e.g. economic models to answer your questions. The aspects of analysis which we do hope to cover here are automated and large-scale analysis, showing tips and tricks for getting and using data, and having a machine do a lot of the work, for example: network analysis or natural language processing.</li>
+<li><strong>Presentation</strong> data only has impact when it is packaged in an appropriate way for the audiences it needs to aim at.</li>
+</ul><p>As you model a data pipeline, it is important to take care that each step is well documented, granular and - if at all possible - automated. This is particularly true when processing more complex datasets, such as transactional government expenditure which has been scraped from a government website.</p>
+<div Class="objavi-no-page-break"><h2>Data provenance</h2>
+<p>Good documentation on data provenance (the origin and history of a dataset) can be compared to the chain of custody which is maintained for criminal investigations: each previous owner of a dataset must be identified, and they are held accountable for the processing and cleaning operations they have performed on the data. For Excel spreadsheets this would include writing down the steps taken in transforming the data, while advanced data tools (such as Open Refine, formerly Google Refine), often provide methods of exporting machine-readable data containing processing history. Any programs that have been written to process the data should be available when users access your end result and shared as open-source code on a public code sharing site such as GitHub.</p>
+</div><div Class="objavi-no-page-break"><h3>Tools for documenting your data work</h3>
+<p>Documenting the transformations you perform on your data can be as simple as a detailed prose explanation and a series of spreadsheets that represent key, intermediate steps. But there are also a few products out there that are specifically geared towards helping you do this. Socrata is one platform that helps you perform transforms on spreadsheet-like data and share them with others easily. You can also use the Data Hub (pictured below), an open source platform that allows for several versions of a spreadsheet to be collected together into one dataset, and also auto-generates an API to boot.</p>
+</div><div id="lightimage"><img src="static/Screen%20Shot%202012-11-15%20at%2015.35.38.png" alt=""></div>
+<div Class="objavi-no-page-break"><h3>Normalizing data</h3>
+<p>Data that comes from the government is often generated across multiple departments by hand. This can result in inconsistencies in what kinds of values or formats are used to describe the same meaning. Normalizing values to be consistent across a dataset is therefore a common activity.</p>
+</div><p>First, you want to start by finding all of the distinct ranges of values for the different columns in your dataset. You can accomplish this by using a database query language (such as SQL's DISTINCT), or by simply using the 'filter' property on a spreadsheet program.</p>
+<p>For example, if you have a spreadsheet with contracting data, and one column is 'Competed?', you would expect the values to be 'yes' or 'no'. But if this spreadsheet is an amalgam of spreadsheet data from multiple users and departments, your values could vary among the following: 'Y', 'YES', 'yes', 1, 'True', 'T', 't', 'N', 'NO', 'no', 0, 'False', 'F', 'f', etc. Limiting all of these potential values to two clear options will make it easier to analyse the data, and also easier for those who follow in your footsteps.</p>
+<p>Especially with financial data, numbers can be formatted several different ways. For example, are your negative values represented with a '-' or placed inside '( )' or possibly even highlighted in red? Not all of these values will be easily read by a computer program (especially the color), so you'll want to pick something clear and consistent to convert all your negative values to (probably the negative sign).</p>
+<p>Is all your numerical data measured out in ones or is abbreviated in thousands? Especially with budget data, order of magnitude errors are not uncommon when one department thinks they're reporting in thousands or millions by default but others expand their data all the way to the ones place. Are some values in scientific notation (e.g. 10e3938)? Make sure all your values are consistent, otherwise your analysis could contain serious errors.</p>
+<div id="lightimage"><img src="static/Screen%20Shot%202012-11-15%20at%2011.34.49%20AM.png" alt=""><em>A column of data requiring name normalization</em></div>
+<div Class="objavi-no-page-break"><h2>Data quality</h2>
+<p>Data cleaning and data quality go hand in hand. You may find it helpful to conduct data quality tests after your analysis in order to measure your confidence in the data, or before to decide whether you want to do the analysis at all. Data quality can mean a few different things but here or some key factors to consider.</p>
+</div><div Class="objavi-no-page-break"><h3>Completeness</h3>
+<p>Is all your data there? If you have a time series of some data, do you have approximately the same number of rows for each year or are there major outliers? An extremely high or low number of rows that looks out of place may indicate duplicate or lost rows. Or it could indicate a change in how data was reported at that time. Are there values for each column where values are required (according to the documentation)?</p>
+</div><div Class="objavi-no-page-break"><h3>Accuracy</h3>
+<p>Does your dataset match up with other available data out there? This can be difficult to confirm, but it is often useful to employ a sampling methodology. Select a significant sample of your records (3-5%) and decide which values you can check for accuracy. If your data is transactional contract data, you could request the full contracts for your sample records and compare the structured data you have with the prose of the contract to confirm that it is correct. You can sometimes check contract and grant amounts from journalistic outlets and even company websites.</p>
+</div><div Class="objavi-no-page-break"><h3>Timeliness</h3>
+<p>Sometimes governments can lag behind in their reporting. Even if a dataset says it's for a specific time period, do you notice a long tail of records closer to the end of the time frame? If so, you may want to restrict your analysis to the subset of data where you have a healthy sampling.</p>
+</div><div Class="objavi-no-page-break"><h3>Case studies</h3>
+<p>Some projects have gone beyond simple assessments of data quality to ensure confidence in the underlying result of a project, but are projects completely about data quality themselves!</p>
+</div><p><em>The Open Knowledge Foundation recently created a data quality monitor for UK transactional expenditure data (<a title="http://opencorporates.com" href="http://openspending.org/resources/gb-spending">http://openspending.org/resources/gb-spending</a>) which has been developed in cooperation with the data.gov.uk team. Each imported file is documented, highlighting any mistakes in the data structure or formal representation of the data. More information at:</em></p>
+<p><em>The Sunlight Foundation has run a yearly report called Clearspending (<a title="http://opencorporates.com" href="http://sunlightfoundation.com/clearspending/">http://sunlightfoundation.com/clearspending/</a>) for the past three years that assesses the quality of the data reported in the US federal spending dashboard, USASpending.gov. The results haven't been very good but have highlighted the need for data standardization and oversight of the data release process.</em></p>
+<p>As an even more thorough technique, little programs and tests can be written to check data validity along the processing pipeline by testing certain assumptions about each record in the dataset. If you do not have the option of creating such tests, you may still be able to create a spreadsheet in which you record known defects and issues with the data, including source files that did not open and inconsistent column values.</p>
+<p>The most powerful data quality tool is much more simple, though: sharing the data. Without access to your source data, others will have to trust your word blindly. Therefore, sharing both the source and processed forms of your dataset is essential to discussing data quality.</p>
+<div Class="objavi-no-page-break"><h2>Categorization and reference data</h2>
+<p>One of the most powerful ways of making data more meaningful for analysis is to combine it with <em>reference data</em> and <em>code sheets</em>. Unlike transaction data - such as statistical time series or budget figures - reference data does not describe observations about reality - it merely contains additional details on category schemes, government programmes, persons, companies or geographies mentioned in the data.</p>
+</div><p>For example, in the German federal budget, each line item is identified through an eleven-digit code. This code includes three-digit identifiers for the functional and economic purpose of the allocation. By extending the budget data with the titles and descriptions of each economic and functional taxonomy entry, two additional dimensions become available that enable queries such as the overall pension commitments of the government, or the sum of all programmes with defence functions.</p>
+<p>The main groups of reference data that are used with government finance include code sheets, geographic identifiers and identifiers for companies and other organizations:</p>
+<div Class="objavi-no-page-break"><h3>Classification reference data</h3>
+<p>Reference data are dictionaries for the categorizations included in a financial datasets. They may include descriptions of government programmes, economic, functional or institutional classification schemes, charts of account and many other types of schemes used to classify and allocate expenditure.</p>
+</div><p>Some such schemes are also standardized beyond individual countries, such as the UN's classification of functions of government (COFOG) and the OECD DAC Sector codes (<a title="http://opencorporates.com" href="http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm">http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm</a>). Still, the large majority of governments use their own code sheets to allocate and classify expenditure. In such cases, it is often advisable to request access to the code list versions used internally by government, including revisions over time that may have changed how certain programmes were classified.</p>
+<p>A library of reference data that can be re-used across different projects and it is a valuable asset for any organization working with government finance. Sharing such data with others is crucial, as it will help to enable comparable outputs and open up options for future data integration. Existing repositories include the IATI Standard (<a title="http://opencorporates.com" href="http://iatistandard.org/">http://iatistandard.org/</a>) and datahub.io.</p>
+<div Class="objavi-no-page-break"><h3>Geographic identifiers</h3>
+<p>Geographic identifiers are used to describe administrative boundaries or specific locations identified in a dataset. While some regional classifications (such as the EU NUTS) are released on the web, there is also an increasing number of open databases which contain geographic names - including geonames.org and the recently developed world.db.</p>
+</div><p>Another related technique is the process of reverse geo-coding: translating a human-readable address into a pair of coordinates. Services like nominatim (<a title="http://opencorporates.com" href="http://nominatim.openstreetmap.org/">http://nominatim.openstreetmap.org/</a>) will not only enable users to generate precise maps of projects in a region, they will also return the responsible administrative boundary for many coordinates. This means that projects which are given by precise address can also be aggregated by state, region or any other geographic unit.</p>
+<p>Additionally, many countries have shapefiles of their political and geographic districts available (usually through the census or interior bureaus) that can be imported into custom mapping applications, like TileMill (<a title="http://opencorporates.com" href="http://mapbox.com/tilemill/">http://mapbox.com/tilemill/</a>).</p>
+<div Class="objavi-no-page-break"><h3>Company and organisational identifiers</h3>
+<p>As you look into spending data that includes recipients outside the government, you'll find companies which act as suppliers to government, but also other types of entities including charities, associations, foreign governments, political parties and even individuals which act as recipients of direct assistance.</p>
+</div><p>Identifying such entities is notoriously hard, since the only information kept by government is often a simple name (which may not uniquely identify the beneficiary, for example "MS"). While most (if not all) countries maintain company registers which assign some type of unique identifier to a company, these databases are often not accessible in bulk and not used coherently across different parts of government. Alternative identifiers - such as tax identifiers and company IDs from private business information suppliers (such as Dun &amp; Bradstreet in the US) - further complicate this process.</p>
+<p>As an alternative, open registries are beginning to compile organisational identifiers in a form that is easy to re-use and thus enables the sharing of databases which have been augmented with such information. OpenCorporates.com (<a title="http://opencorporates.com" href="http://opencorporates.com">http://opencorporates.com</a>) is a startup that collects information from companies world-wide and provides a convenient API to match datasets with the list of known countries. The IATI project for aid transparency is working towards similar standards for other organisations, such as foreign governments and charities active in the development space.</p><h1 id="NetDspqLaA_8"><strong Class="initial">7.</strong> Analysis</h1>
+<p style="text-align: center;"><img src="static/pipeline-analysing-data.jpg" alt=""></p>
+<p style="text-align: center;"><em>Data Processing Pipeline</em></p>
+<p style="text-align: center;"> </p>
+<div>Dear reader - we know you are the experts. We're not going to teach you all the basics, but this section's aim is to show you how technology can help speed up, scale up or simplify your working practices. For this reason in this section we will say only a few words about statistical methods, even though many of you will be very familiar with those.</div>
+<p>Once you have your data in a suitable format to be dealt with, much of your analysis can be automated with a custom program or inside spreadsheet software. Obviously the deeper your data dive is, the less automated things become, but having a standard set of post-processing, analytical steps to run on a dataset helps make sure you haven't missed anything. Some of the steps may seem basic, but it always helps to have a checklist!</p>
+<p><strong>Sorting data</strong></p>
+<p>Possibly the simplest way to find some interesting points is to sort by a column or category. In spending datasets, it's common to rank data by the largest transactions and look at the beneficiaries. This also gets you the maximum or minimum values of a column, which could be interesting.</p>
+<p><strong>Basic statistical analysis</strong></p>
+<p>Statistical analysis such as calculating percentiles or the average of a dataset, are essential to characterize your data. These are available in most programming languages as well as most spreadsheet programs like excel. A more complete list of statistical tools and resources can be found at the resources chapter.</p>
+<ul><li>An <strong>average</strong> is a measure of central tendency in a sample. It can be described as the "typical" or representative value of a data set. Knowing the average helps detect whether any single payment is atypical.</li>
+<li>A <strong>percentile</strong> is the value of a variable under which a certain percentage of the data fall. For example, if a transaction amount is in the 90th percentile, then it is larger than 89% of the transactions in the dataset.</li>
+<li>A <strong>standard deviation</strong> measures how far away a value is from the average. If your standard deviation is low, then it means your row values are clustered together. If it's high, then they are spread out. By checking for values that are one, two, or three standard deviations outside the mean, you can detect outliers. Outliers could be errors or extremely interesting data points.</li>
+</ul><p><strong>Summarizing or aggregating by categories</strong></p>
+<p><strong></strong>Sometimes data is available at a very high level of detail, making it hard to see the big picture. It's a good idea to have a standard set of categories to aggregate by, such as geographic unit, functional classification, account number, fiscal year or other temporal unit, government department, or even recipient.</p>
+<p><strong>Normalization for comparability</strong></p>
+<p><strong></strong>In order to be able to compare data across time or other parameters, you may need to systematically adjust the data values. Here are some examples.</p>
+<ul><li><strong>Deflation or inflation</strong>. Expenditure data is naturally obtained with current (or nominal) money value, but such data is not directly comparable across different years since inflation affects the value of money. In order to compare currency across years, it is necessary to deflate or inflate the value according to a base year. For example if we wish to compare the budget of a program in dollars from 2000 with the same program in 2012 we must adjust the value of the 2000 data and the 2012 dollars to a base year. Deflator and inflator factors are calculated by central banks in most countries and a comprehensive list by country can be found at the World Bank website (<a title="http://worldbank.org" href="http://worldbank.org">http://worldbank.org</a>)</li>
+<li><strong>Scale and normalize values</strong>. When we wish to compare patterns of budget programs sometimes the difference in scale between budgets may prove difficult to display in a graphic form. In such cases is useful to compare percentages instead of absolute numbers. For instance, it is very common to express budget and spending data as percentages of a country's GDP, instead of in absolute numbers. If you're mapping spending, make sure to scale your data by population density. Otherwise, your map will most likely just be a map of where your populace lives.</li>
+</ul><div id="note">You may want to visualize your data to get better understanding of your initial assumptions. This may save you a lot of time in case your initial assumptions were not good, and may also help you to get more insight into new findings your data might contain but previously were not clearly visible. For this purpose you can use scatter plots, line graphs, heat maps, bar charts.</div><h1 id="NetDspqLaA_9"><strong Class="initial">8.</strong> Using technology in your work</h1>
+<p>As you move into more advanced data projects, you may find yourself in a situation where a research project turns into an effort that involves coders, designing databases and web sites. It is important that you take a step back and realize that you are now running not just an advocacy effort, but also an IT project.</p>
+<p>There are many difficulties which CSOs face when developing software. Some common issues include:</p>
+<ul><li>Difficulties in finding qualified developers that want to contribute to your projects at a reasonable rate, as well as in the communication between CSO staff and developers.</li>
+<li>Clearly communicating the requirements for the software so that both non-technical and technical staff share a vision for the outcome of the project.</li>
+<li>The estimation of time and resources for particular tasks, especially how to handle projects that drastically overrun the timeframe and funding they were initially assigned.</li>
+<li>Evaluating the work of developers to ensure that the product that has been delivered is according to what has been agreed, especially in small projects with only a single coder or when working with external contractors.</li>
+<li>Maintaining the project after the main development period has finished</li>
+</ul><div Class="objavi-no-page-break"><h3>Starting from scratch vs. re-using Components</h3>
+<p>It is always going to be more costly and riskier to develop something from scratch than to customise something that already exists.</p>
+</div><p>You should make the software behind your projects open source. If many other organizations do the same, this allows code to be reused across jurisdictions. Not only does this ease the financial burden, but it helps create the expectation in populaces around the globe for the high quality engagement tools that their neighbouring country has access to.</p>
+<p>That's not to say you should never develop something new - just ask around first, and make sure that what you are asking is technically feasible.</p>
+<div Class="objavi-no-page-break"><h3>Commissioning New Software</h3>
+<p>Purchasing software is more closely related to having a piece of clothing made than to buying chairs. You can give the designer a basic vision of what you would like, but you will always need to come back to make sure it really fits, and your thoughts may change when you see things in practice. If you have an arrangement with your tailor which allows you to first specify the general idea, and a couple of other appointments for fittings and trials, you'll probably end up with a better and more creative result than if you tried to design the whole thing and it was simply unveiled to you at the end. You'll also feel more in control and it may even be quicker to do design and implementation in parallel.</p>
+</div><div Class="objavi-no-page-break"><h3>Defining requirements &amp; Designing the solution</h3>
+<p>Define the basic components of your project and prioritise them by their importance. As the developers start working on one of these chunks, you can then break it up into more specific tasks based on your evolved understanding of the project. A popular technique for this purpose are <em>user stories</em>, small narrative pieces that describe each problem: "As a [web site visitor] I want to [be able to see a supplier's contracts] so that I can [understand what services they provided to government]". The key to these stories is that they describe the actual user need, not the details of the solutions that you have envisaged. While you should of course discuss those with the developers as well, defining solutions is mainly the job of the developers, not the project manager.</p>
+</div><div Class="objavi-no-page-break"><h3>Implementation: iterations and milestones</h3>
+<p>A saying amongst developers goes: "Walking on water and developing software from a specification are easy if both are frozen." As your software project is progressing you will likely realize that the specifications you have given need to be revised or extended. Yet by modifying the requirements you are essentially shifting the ground on which developers are executing - meaning they will have to stop their work to adjust. To prevent such changes from freezing all development, the process of introducing changes and additional requirements needs to be structured.</p>
+</div><p><em>Iterations</em> are periods of a defined length - often two or four weeks - during which developers are tasked to execute a set of previously selected user stories or requirements. Before the iteration starts, developers have to pull in the work from a list of tasks (a so-called <em>backlog</em>) prepared by the project manager, committing themselves to delivering those tasks within the agreed period. Crucially, project managers are not allowed to extend or revise the scope of an iteration while it is ongoing (unless they want to declare it failed). This method ensures that changes are introduced in bulk and understood by the team. This approach mandates the opposite of the more common unstructured communication between managers and developers, e.g. emails with unsorted lists of change requests which tend to be ignored and lead to confusion.</p>
+<p>Whenever you consider an additional requirement, be sure to consider if it is realistic within the resources you have available. "Scope creep", the progressive extension of a project during its development, is a common cause of project failure. By becoming more and more ambitious, the project finally ends up with no usable product at all. To avoid scope creep, make sure to have a storage area for long-term ideas. Also make sure that developers accept additional tasks through a pull process, and not by having them pushed into their workflow.</p>
+<div Class="objavi-no-page-break"><h3>Maintenance</h3>
+<p>Make sure to budget a for ongoing maintenance after the end of your project. Who is going to guarantee that the servers stay online? Who is going to fix a typo? It is unlikely that your project will remain entirely static after its initial development, so you should have an explicit agreement with the developers regarding future support. It is also useful to collect feedback after the projects release to commission a small number of additional days when enough additional work has accumulated.</p>
+</div><div Class="objavi-no-page-break"><h2>Roles and how to find developers</h2>
+<p>The key ingredient to a successful software development project is having the right people on staff or as contractors. Depending on the scope and type of your project, you may need a variety of skills - these are some of the common descriptions:</p>
+</div><ul><li><strong>Web designers</strong> typically produce designs and layouts for web pages, often initially in a graphics program like Adobe Photoshop. Most, but not all, web designers then translate their designs into web markup (HTML, CSS).</li>
+<li><strong>Web developers</strong> are more technical. They produce interactive web interfaces such as search masks, browsers or specific form-based operations. They often use programming languages such as PHP, Ruby on Rails, Python or JavaScript.</li>
+<li><strong>Visualization designers</strong> develop graphics that represent quantitative information. A key distinction here is between non-interactive graphics (i.e. static images) and interactive visualizations, which often require some programming. There are still very few designers who design interactive visualizations, so rates may be relatively high.</li>
+<li><strong>Software developers</strong> are even more technical, developing backend software for data processing or acquisition. They are experienced in the use of database software (such as SQL databases) and programming languages such as Ruby, Python or Java.</li>
+<li><strong>Data scientists</strong> and <strong>statisticians</strong> produce analysis based on large sets of data, detecting tendencies and outliers in the dataset. They are not usually expected to produce front-end applications, but may produce software in the process of analysing data.</li>
+<li><strong>Usability experts</strong> and <strong>user experience (UX) designers</strong> think about the way your user will interact with your site answering questions such as 'is it obvious from the landing page what the purpose of this site is?'</li>
+<li><strong>Testers</strong> try and break things to test their robustness. This is particularly useful e.g. if you think your project will receive a lot of traffic as a result of a media campaign, you want to know your site can survive the hit</li>
+</ul><div Class="objavi-no-page-break"><h3>Good places to look for developers</h3>
+<p>The easiest way to meet developers is through community meetups, such as <em>hackdays</em>. During such events, coders meet up to cooperatively develop prototypes of new software. To meet volunteer developers who can help you make sense and unleash the power of government spending and budgets, it's wortwhile to investigate events such as Random Hacks of Kindness (<a title="http://techcampglobal.org/" href="http://www.rhok.org/">http://www.rhok.org</a>), Data Kind (<a title="http://techcampglobal.org/" href="http://datakind.org/">http://datakind.org/</a>) and TechCamps (<a title="http://techcampglobal.org/" href="http://techcampglobal.org/">http://techcampglobal.org/</a>).</p>
+</div><p>There are a few ways you can discover if there is a hackday in your area. One way is to search on Lanyrd (<a title="http://techcampglobal.org/" href="http://lanyrd.com/search/?q=hackday&amp;context=future">http://lanyrd.com/search/?q=hackday&amp;context=future</a>) or set up an account on that system and request that you are alerted when there is a hackday in your area. Another approach is join mailing lists of organisations that might help you find developers e.g. the Open Knowledge Foundation lists (<a title="http://techcampglobal.org/" href="http://lists.okfn.org/mailman/listinfo">http://lists.okfn.org/mailman/listinfo</a>) or the Sunlight Labs mailing list (<a href="https://groups.google.com/forum/#!forum/sunlightlabs">https://groups.google.com/forum/#!forum/sunlightlabs</a>).</p>
+<div Class="objavi-no-page-break"><h3>What do things cost?</h3>
+<p>It's impossible to give concrete guidelines on how much a project should cost. Developers' salaries are generally quite high for a country's average, but vary very strongly from country to country. Worried about your project spiraling out of control? We'd recommend agreeing on a price per iteration, and it may be a good idea to draw up a contract which allows you to break it off if you are not happy with the work at the end of an iteration. Plus, you can generally also find a friendly developer to glance over a quote from a company for a sanity check.</p></div><div id="section-3" Class="objavi-subsection"><div Class="objavi-subsection-heading">Presentation and engagement</div><div Class="objavi-chapter"><strong Class="initial">9.</strong> Presentation and engagement</div><div Class="objavi-chapter"><strong Class="initial">10.</strong> Selecting methods and tools</div></div><h1 id="NetDspqLaA_10"><strong Class="initial">9.</strong> Presentation and engagement</h1>
+<p>While raw financial data is valuable just as a tool for research, analysis and investigation, it can also be directly presented to an audience in a wide variety of ways. Such presentations can take the form of press releases in which data is used to make an argument, interactive infographics which condense information to visually represent a key finding or even searchable on-line databases that allow each user to find spending in their neighbourhood.</p>
+<p><img src="static/pipeline-presenting-data.jpg" alt=""></p>
+<div Class="objavi-no-page-break"><h2>Clarifying your topic</h2>
+<p><em>"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away" - Antoine de Saint Exupéry</em></p>
+</div><p>If you are reading this, you probably already have a fairly clear idea of the topic that you want to investigate and present. Still, it is useful to look at different ways of defining your subject. In some cases, you may merely want to give an overview of a particular area, e.g. by showing the proportions of government spending which go into the various functions of government. In other cases you may have a particular hypothesis that you want to portray or a particular journalistic story that you mean to tell, such as a case of systemic corruption that becomes evident only as you contrast one department's spending patterns with that of another one.</p>
+<p>Sometimes there is a temptation to get lost in complexity. Using all the data and tools available to you to their full potential can force an almost endless amount of information onto our audiences. The benefit of doing so is limited, however: people will not understand the point that you are making, or, even worse, ignore you.</p>
+<p>At the same time, technology can also be used to do the opposite: focus in on the key points, hiding other detail. Selecting where to go into depth doesn't have to be the author's responsibility any longer, it can be a choice that the user of data-driven applications is empowered to make based on their own interests. The prerequisite for using technology to create a clear focus is answering the question: What is the core of what we mean to present, what is the essence of the narrative our project is intended to convey?</p>
+<div Class="objavi-no-page-break"><h2>Setting an objective</h2>
+<p>As you focus the topical core of your project, you also need to think about the objectives of your project: is it intended to inform the general public about an aspect of government budgeting or spending practices? Are you making a specific case to government, e.g. by proposing a new policy which you have gathered evidence and developed arguments for?</p>
+</div><p>Your project can also serve as a forum for discussions, which would usually require you to facilitate the debate to maintain ongoing engagement.</p>
+<p>The objective may be to establish a network-based, collaborative effort, e.g. to build a community that may review documents released by the government or evaluate the conditions of public contracts. Such modes of engagement require extensive design to encourage users to contribute in a way which best matches their particular skills and interests.</p>
+<p>Some example of working with the public to collaboratively clean up financial data is Linked Gov (<a title="http://yourbudgit.com" href="http://linkedgov.org/">http://linkedgov.org/</a>). This is a platform that will allow the public and civil servants to access and do small tasks that will clean government financial data.</p>
+<div Class="objavi-no-page-break"><h2>Targeting an audience</h2>
+<p>The efficiency of projects based on spending data depends heavily on how well you have defined and understood your audience. If your message is too general for a specialized audience, it will be ignored. At the same time, a complicated and very technical presentation will usually fail to address broader parts of the population. Whether it be that your main audience is a specific constituency, government officials, the general public or a combination of those, it is important that the media and methods you apply help to get the message(s) across to the audience that you are targeting.</p>
+</div><div Class="objavi-no-page-break"><h3><strong>The general public</strong></h3>
+<p><strong></strong>Is the purpose of your campaign to stimulate discussion amongst the general public? Unless your organization has exceedingly well-developed outreach channels, you will not usually be able to address the general public directly. Still, thanks to the social media and internet-based direct communication, many organizations can now contact and involve citizens who have shown some interest in their work directly. In cases where your main audience is the general public, simplicity is a must. It is necessary that the vast majority of people get a grasp of what you are trying to communicate. In these cases you may need to emphasize the visual presentation of the data that gets quickly and succinctly to the point, such as infographics. See the case study at the end of this chapter for how an organisation in Nigeria used Social Media to reach out to the general public.</p>
+</div><div Class="objavi-no-page-break"><h3><strong>Mass media</strong></h3>
+<p><strong></strong>Unlike direct channels, broadcast and print media require you to first convince an intermediary of the relevance and quality of your message. Media are often the best channels to use when we want to create social pressure on governments to change some budget issue.</p>
+</div><div Class="objavi-no-page-break"><h3>Community and constituency</h3>
+<p><strong></strong> This may include people and organizations with which you have previously cooperated or which are active in the same domain. Unlike the wider public, they are more likely to be interested not just in your conclusions, but also in the data and tools that you have used to reach your conclusions. Creating an open space for your community to discuss your work is essential in getting relevant feedback. When you need to reach to your constituency it may be necessary to provide a broader context, such as a full report including methodology, conclusions and possible paths of further discussion.</p>
+</div><div Class="objavi-no-page-break"><h3>Government</h3>
+<p>Officials are often a target of your outreach, since they are in the position of actually deciding and implementing changes in policy. While they can be reached through public pressure and the media, your organization or some of its constituency may also choose to contact governments directly. In the case of trying to reach government officials, a balance of the previous approaches is very important. Meetings with government officials are rare and short-lived. Condense your main argument and supporting evidence into one page or less, and include a clear action that you want them to take.</p>
+</div><p>Obviously, on any given issue you may choose to involve any set of these groups, but it is still important to have a clear picture of who they are and what their incentives motivate them.</p>
+<p><img src="static/pyramid_1.jpg" alt=""></p>
+<div Class="objavi-no-page-break"><h2>Case study</h2>
+<p><em>BudgIT Nigeria on social media</em></p>
+</div><p>Data is invaluable. To the uninformed person where information asymmetry thrives, access to data illuminates the path to facts and provokes emotions that trigger results. Nevertheless, poor handling of data puts valuable facts in an opaque structure that communicates nothing. Data could be in a maze of thick document riddled with complex terms or iterations of figures that doesn’t connect the user. Not being able to promote discussion nor give provide contextual understanding, data may could be worthless.</p>
+<div id="lightimage"><img src="static/nigeria-budget2.png" alt=""></div>
+<p>Nigeria returned to democracy in 1999 after lengthy years of military rule. Data under the barrel of the gun was an exclusive preserve of rulers as probing the facts behind data is taken as an affront to authority and strive to question the stained reputation of the junta. Civil servants were bound by the Official Secrets Act not to share government information hereby putting the citizens in the dark. Even after thirteen years of return to democracy, there is clearly a gap in accessing public data with some government officials still stricken with military era hangover. Data especially in terms of public expenditure communicates little to the larger sections of public who are not versed in financial accounting nor have requisite understanding to evaluate the complex arithmetic.</p>
+<p>BudgIT, creative start-up sees a huge opportunity in using creative data visualization to stimulate interests concerning public expenditure. Understanding the ubiquity of the mobile device within the Nigerian locality and the increasing number of Nigerians online, BudgIT sees the opportunity to engage Nigerians and explain public expenditure in a simpler way. This thrives on building engagement across all platforms, encouraging the community to action via NGOs and reaching out to everyone to promote citizen participation. This is about making public data a social object and building an extensive network that demands change.</p>
+<div Class="objavi-no-page-break"><h3>Building The Community</h3>
+<p>Across our interactions with users, we see a gulf in understanding what the budget is and what the citizens usually expect. We have engaged over 10,000 Nigerians over the budget and we profile them into three to ensure optimum value is delivered. The categories are briefly explained below:</p>
+</div><p>Singular Users: These are users who want information in simple and quick format. There are interested in data but not in a deeper scale of analyzing the content or probing further. A series of simple tweets is enough for them to retell the stories or interactive applications that give a snapshot.</p>
+<p>Closed Loop Users: A set of users who stimulate a discussion pattern, engage the data channel effectively to increase their knowledge of subject matter or challenge the assumptions of data. These set of users are adherents of the platform via social media, hence they ensure in building the platform through feedback systems or referral to their social connections.</p>
+<p>The Data Hog: This set of users want the raw datasets to rework visualization or do extensive analysis for personal or enterprise purposes. We simply give them the data for their definitive purposes.</p>
+<div Class="objavi-no-page-break"><h3>Engaging the Citizen</h3>
+<p>Across every society lies a literacy span and engaging every component is highly critical to the societal growth and stability. There is always a storyboard in every life that volumes of data can be matched with. People constantly want to be more informed especially concerning issues that they find difficult to understand. Engaging citizens is to take critical analysis of the target users at a time and itemize the possible profiles. A look at user profile demands a thorough analysis of their empathy, attention and insight towards the data available to them. What does the Nigerian citizen care about? Where is the information gap? How quickly can we reach out to them and place data in the storyboard of their lives? A critical understanding of the user’s psychology and the perceived response to the data is the first needed level of analysis. BudgIT’s immediate reach is to the average literate Nigerian connected to online forums and social media. Most online users amidst the array of interests in gaming, reading and sharing social connections within a limited timeframe will definitely need data in a brief and concise manner. After a snapshot of data either as a tweet or infographics, there’s an opportunity to build linked data on other platforms where the big picture can be set and interaction can be enhanced. &lt;p &gt;An important angle of visualization to us is understanding the data appreciation level of the users. Complex diagrams, superb infographics and aesthetic interactive applications might not convey the exact meaning to user based on his/her previous approach to data. Data vizualization needs to take into consideration how users can easily grasp the vizualized data and subject it to personal interpretation. A good visualization transfers knowledge and mostly important brings forth a story the user can easily connect with.</p>
+</div><p>For us in BudgIT, our engagement model is anchored on the following:</p>
+<ul><li>Stimulating discussion around Trends: In engaging with users on public data, BudgIT keeps track of online and offline discussions and seeks to provide data to enliven the interactions. A glaring example was the fuel strikes in January 2012 where there was a constant agitation among the protesters on the need to reduce the size of governance. BudgIT tracking the discussion via social media in 36 active hours quickly built an app that allows citizens to cut the Nigerian budget. The huge response with over 3,000 users who interacted with the budget using the app refined our engagement model. We keep looking for trends in the polity and matching it with relevant data quickly rendered into tweets or infographic display that quickly  extends our influence.</li>
+<li>Constructive feedback mechanism and Balanced outlook: Data speaks volumes and individuals subject it to personal interpretations. In the engagement with users, feedback is enabled through discussion boards or retweets. Most users throw up discussions that tend to ask about stories behind the data and seeking opinions of BudgIT. Its of utmost priority to ensure that opinions only explain the facts behind the data and does not conform to individual disposition to the subject matter. It’s most necessary to build up feedback channels and engage the users creatively to ensure the community built around the data is sustained.</li>
+<li>Localize Outlook of Data: For a dataset targeted at a particular group, BudgIT is building competency to localize its content and promote a channel of discussion that connects to the users. This involves taking a cultural outlook at the icons, symbols, objects and language to ensure that engagement concerning the budget flows seamlessly. This phase of engagement is with the grassroots who mostly don’t have access to Internet but only possess SMS-based mobile phones.</li>
+</ul><p>After making the public expenditure data available in an easy-to-read format as shown on our portal (<a title="http://yourbudgit.com" href="http://yourbudgit.com">http://yourbudgit.com</a>), we reach out to the citizens through the civil society organizations to ensure that citizens monitor capital projects in the budget. We also plan to develop a participatory framework where citizens and government institutions can meet in town halls to define key items in the budget that needs to be prioritized. Once we get citizens to be aware of capital projects in the budget and connect them with civil societies where BudgIT is not located, citizens can track report projects and report status. Ensuring that citizens of any literacy span are armed with data and possess a clear path to demand action, BudgIT is crossing the rubicon from open data to open action.</p><h1 id="NetDspqLaA_12"><strong Class="initial">10.</strong> Selecting methods and tools</h1>
+<p>The key to using data to tell people about your issue is to choose a really good story. The key to finding a good story is to formulate the right question for your audience.</p>
+<p>Be guided by what interests you, as this stands a good chance of interesting other people. If you don't feel confident about your judgement then try lots of things that you consider interesting and see if it makes impact on the people you care about. Are they sharing your work on social media? Do they email you or link to your work from their own sites? If the answer is yes then you are really making good data stories.</p>
+<p>Of course rigour and accountability are essential and you should always include links to your methodology and data, but it doesn't need to obscure your headline. Telling a story is the difference between people not reading your work and making it all worth the effort.</p>
+<p>If you've already got data products that you want to share, try disseminating them in a couple of different ways. In addition to the methods outlined below, try preparing primers or manuals for other groups that want to start working with datasets that you may be particularly experienced with. You can also organize workshops, meetups, and webinars to familiarize groups with the data you work with. If grassroots issue groups can internalize your data products and present them in a contextually relevant way to their user base, then both groups have gained something from the partnership.</p>
+<p>This is a collection of (hopefully) inspiring examples where groups have connected to the public directly through their websites, social media, or through the mainstream media. If you are interested in making media but don't have the capacity then fear not! Check out the first section of the book on reaching out to other organizations for help or running your own project.</p>
+<div Class="objavi-no-page-break"><h2>Communicating via the web</h2>
+<p>If you are creating or adding to a website that you want citizens to access directly then here are some of the ways you can really use budgets or spending data to increase the value of your organisation.</p>
+</div><p>Firstly we highly recommend that if you have some really great data, that's a story in itself, then don't waste any time - put the data up on your website and write a great story to go with it. Make sure the full dataset is available to download, ideally in a couple of formats like <em>csv</em> and <em>json</em>, and explain your methods. Then encourage your supporters to use and play with the data, add comments and talk in your community site or social networks.</p>
+<p>If you feel your data needs that little bit more work to make it interesting or digestible, here are some examples of portals, interactive infographics, images, reports, and a searchable dataset.</p>
+<div Class="objavi-no-page-break"><h3>Infographics and images</h3>
+<p>The art of the infographic is to make the complex look simple. This can be invaluable if you can communicate huge amounts of data in one picture. It is always recommended, both as a boost to your credibility and to allow others to extend your work, to publish the complete data needed to recreate the image. This allows your community of supporters to create their own visualisation and check your work easily before republishing it.</p>
+</div><p>Below is an infographic summarizing the history of the US Federal Budget, by the <a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.cbo.gov/publication/42636">Congressional Budget Office (CBO)</a> /<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.cbo.gov/publication/42636">http://www.cbo.gov/publication/42636</a>). It shows the budget over time and as a percentage of GDP, and in the context of total spending and revenue. This was used to emphasise that the federal government's budget deficit for 2011 was the third-largest in the past 40 years:</p>
+<p><img src="static/budgetinfographic2.png" alt=""></p>
+<div Class="objavi-no-page-break"><h3>Images of spending</h3>
+<p>This "binder full of burgers" infographic was created to show the sandwich of spending for Barack Obama's 2012 presidential election campaign. The data is not especially novel but the graphic is fun and engaging.</p>
+</div><p><img title="burger full binders" src="http://okfn.booktype.pro/spending-data-handbook/_edit/static/tumblr_md0vl73RzF1rkwn28o1_1280.jpg" alt="burger full binders"></p>
+<div Class="objavi-no-page-break"><h3>Interactive infographics</h3>
+<p>An interactive infographic can add even more data to a single screen full of images. A simple example is a bar chart where hovering over a bar with the cursor results in a dialogue box that tells more detail, like the exact value of that bar.</p>
+</div><p>It seems there are two types of interactive graphics that work very well. One is where the user finds their own path through a complicated landscape of data. For example a map of the world where you can zoom in to details of each country or regions within a country and discover some facts about that area. The other is where the infographic guides the user through a story. For spending data in particular, people often like the ability to "drill down" in an infographic, or in other words, start with a high level view of the data and then gradually view more detail about different sections that they can select.</p>
+<p>In recent years we have seen the quality of interactive infographics grow and become easier to produce. Anyone with a Google account can now make maps (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.google.com/drive/start/apps.html#fusiontables">http://www.google.com/drive/start/apps.html#fusiontables</a>) and charts (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="https://developers.google.com/chart/">https://developers.google.com/chart/</a>). Equally, the tools to manipulate and display data in more sophisticated ways have become more accessible with efficient and usable<em> javascript</em> libraries. Using these libraries requires a bit of technical skill, so you may want to reach out to other organizations, as described in the first section of this book.</p>
+<div Class="objavi-no-page-break"><h3>Budget interactives</h3>
+<p>An interactive graphic gives you the perfect opportunity to allow people to choose their own budget. This interactive chart called Citizen Budget is from the non-profit Open North. It shows Canadian budgets and gives citizens the chance to play at being the official controller. They can cut spending in any area they choose but the budget has to balance.</p>
+</div><p><img src="http://okfn.booktype.pro/spending-data-handbook/_edit/static/citizen-budget.png" alt="http://citizenbudget.com/"></p>
+<div Class="objavi-no-page-break"><h3>Spending interactives</h3>
+<p>The World Bank have a spending database called Boost. It's a technical collaboration between the World Bank and various governments to digitize their spending data. It is especially targeted at countries that wouldn't otherwise have the technical resources. One of these countries is Kenya and the data has been displayed to great effect here:</p>
+</div><p><img src="static/boost-kenya2.png" alt=""></p>
+<div id="lightimage">The examples given here for interactive infographics showing both budget and spending data are examples of where the user can navigate for themselves. Please let us know if you have examples of step-by-step guides to Budgets and Spending as it would be great to include them here.</div>
+<div Class="objavi-no-page-break"><h3>Online reports</h3>
+<p>A report allows you to expand on points and show lots of different views of a dataset that would be just too cluttered on one infographic or in an interactive. A report is a good place for a step-by-step guide through a more extensive study.</p>
+</div><p>An example of a very short report is from Enough is Enough (EiE), a coalition of Nigerians aiming to enhance citizen engagement and good governance. Their EiE report on the state of the Nation shows a very concise summary of the major spending corruption stories they identified that year, along with some links to sources.</p>
+<div Class="objavi-no-page-break"><h3>Podcasts and videos</h3>
+<p>A well made podcast will almost certainly be shared widely given the right push. Describing the significance of your data and getting expert comments is so much easier to digest in a podcast than a long report - even if they are both very well written. Again, the technology to make radio has radically reduced in price in recent years.</p>
+</div><p>The NPR Planet Money podcast rose to fame for its incredibly accessible, award-winning coverage of the 2008 Global Financial Crisis. Planet money now covers financial issues all over the world on its weekly podcast.</p>
+<div Class="objavi-no-page-break"><h3>Online portal</h3>
+<p>A portal is suitable for a large and relatively frequently updated dataset. People in your community can then build web applications to pull the data. An example of a portal that might be used to add more context to spending data is the police.uk website. This is a complete view, updated monthly, of crimes committed in the UK, with details down to street level. The site also provides an API[15] which you can use to add new data to an app showing, for example, spending on police with crime levels overlaid on map.</p>
+</div><div Class="objavi-no-page-break"><h3>Mobile technologies</h3>
+<p>There are a few free and open tools that will allow you to display your data through a mobile phone or tablet app. AKVO is a non-profit foundation and they describe (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.akvo.org/blog/?p=4822">http://www.akvo.org/blog/?p=4822</a>) how their AKVO FLOW (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.akvo.org/blog/?p=4836">http://www.akvo.org/blog/?p=4836</a>) platform works using free open source software to communicate your data through an Android mobile app.</p>
+</div><div Class="objavi-no-page-break"><h3>Social media</h3>
+<p>Building a community around your cause in general and data analysis in particular is really valuable. Setting up a mailing list and creating a space to collaborate like on a wiki can lead to some really interesting social connections at relatively low cost can allow you to work with some really devoted and knowledgeable people. Some nice examples are the OKFN mailing lists (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://lists.okfn.org/mailman/listinfo">http://lists.okfn.org/mailman/listinfo</a>) using Mail Man - <a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.gnu.org/software/mailman/index.html">http://www.gnu.org/software/mailman/index.html</a>), wikis (for example <a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://www.wikispaces.com/">http://www.wikispaces.com/</a>).</p>
+</div><p>There are also the big social media companies like Twitter and Facebook of course! These are great for reaching large numbers of people who have anything from a passing interest to complete devotion to your cause. When you join create a Twitter account or Facebook page with the name of your organisation. The strategy when joining these is to contact some high profile tweeters or facebookers to take on your cause or message. Make sure there are plenty of links back to your website when you Tweet or post to Facebook or other platforms. If you have a data release then create a package of the data, the story and any other resources (image downloads, reports etc) on a webpage and then tweet a link out with a link to that page. The same principle applies with Facebook.</p>
+<div Class="objavi-no-page-break"><h2>Communicating with the press</h2>
+<p>When writing a press release keep it simple and very relevant to a story the journalist might like to run - leaving descriptions of your organisation or how you got the data to the very end or ideally include only a link to a relevant webpage. To be relevant you need to hook into something in the news cycle (<a title="http://en.wikipedia.org/wiki/24-hour_news_cycle" href="http://en.wikipedia.org/wiki/24-hour_news_cycle">http://en.wikipedia.org/wiki/24-hour_news_cycle</a>) or offer up such a strong story it would create news. If it is the latter, then you might want to give it to a number of newspapers with an embargo so they will release the story all at the same time to create a really big impact.</p>
+</div><p>If you are looking for news hooks then at the simplest level the data can relate to the time of year, maybe when there is a new financial phase or an election and the news is focused very much on spending and budgets. For example if it's a complete cleaned up version of all public spending, or budget or all the pension funds, just showing this data qualifies as something a journalist would find interesting - they may even have an in-house team working on the same thing so will be pleased to have the data quicker. Here the press release can be quite simply describing the data if the data is detailed and relevant enough to stand alone, along with a link to your portal and some instructions for using it, or a download of the data.</p>
+<p>If you're wondering about how much extra work you need to do to present your data before sending out a press release then the same principles apply as when adding content to your website. If you have a great story from a great piece of analysis or just great data alone, then send the data and the story in a concise form with links through to the methodology on your website.</p>
+<p>One final tip before describing what you might include in your press release: when working with journalists be sure to get a link to your website included in the piece.</p>
+<div Class="objavi-no-page-break"><h3>Including images</h3>
+<p>If you do have a dataset with a story that would really benefit from an infographic then add it to the press release. Or if you have a mobile or tablet app, a podcast or report then add a screenshot of the most enticing view of it too. Ideally attach a small copy of the image to the email and give a link to a higher resolution version on your website.</p>
+</div><p>It's likely the newspaper's graphics team will create a version in the newspaper's style so you should include the data for your graphic in a usable format (no PDFs).</p>
+<p>The journalist may also choose to use your graphic as it is, so be sure to add a small discrete copy of your logo in the corner of the image or other citation embedded in the image.</p>
+<div Class="objavi-no-page-break"><h3>Including an interactive infographic</h3>
+<p>Equally you may have made a descriptive interactive, in which case you might want to make it 'embeddable' on other websites. You may also ask for the news site to link back your own website, or include a link in the embeddable portion of the infographic.</p>
+</div><div Class="objavi-no-page-break"><h2>Communicating with governments</h2>
+<p>In modern democracies we would advise CSO's to influence government decision making with evidence-based, factual, easy to understand policy proposals. These proposals can be related to almost any particular subject, from energy efficiency, labour policy, immigration, minority issues to public finance. Government relation to these policy proposals can be decisive in times of elections, and for this reason they cannot ignore them. Government capacity is limited, so you might find it takes a while to get a response from them.</p>
+</div><p>For CSOs it is important to differentiate between Government's general goals, and policy specific goals. For example, every responsible Government has a general goal of maintaining a balanced fiscal policy. As we get more specific about this goal, stakeholders' opinions begin to differ. Governments are rarely very clear about their policy goals, as this would make it easier for CSOs to keep the Government accountable. Real policy goals can also be very different from what Government declares as its policy goals. For example, every Government is against tax evasion, but in many cases Governments indirectly support them, one way or another.</p>
+<div Class="objavi-no-page-break"><h2>Common themes for communicating</h2>
+<p>To summarize, the key themes are: have a great story, be clear, be connected to lots of other people and be accountable by publishing methodology and data. These apply equally if you are communicating with the public or the media.</p>
+</div><p>Being clear is making easily repeated points with the data, or describing a juicy data set in one sentence. If you can make the presentation fun, fascinating, or just beautifully simple then you are on to a good thing.</p>
+<p>Being connected is making things technologically very shareable and then facilitating the spread of your message through your own community of supporters and social media.</p>
+<p>Being accountable is being completely transparent about how you obtained, cleaned and analysed the data. This method has to be explained very clearly too, and the standards maintained for every release until your CSO has a good reputation of trustworthiness. But please don't let concerns about trustworthiness stop your CSO from exploring and having fun with the data in the first place, which is really something you need to consider when settling on a data story.</p>
+<p>The final points are wherever possible, have fun telling your story and don't be afraid to experiment with different styles and tones but don't delay if you have a great dataset on your hands - let the world know!</p><div id="section-4" Class="objavi-subsection"><div Class="objavi-subsection-heading">Appendices</div><div Class="objavi-chapter"><strong Class="initial">11.</strong> Resources</div><div Class="objavi-chapter"><strong Class="initial">12.</strong> Glossary</div></div><h1 id="NetDspqLaA_13"><strong Class="initial">11.</strong> Resources</h1>
+<p>There are a variety of resources that you can use to accomplish some of the technical tasks described in the book. Some are geared towards users who are already experienced in a programming language, and some can be used by those with no experience. Most are helpful with analyzing and visualizing data.</p>
+<div Class="objavi-no-page-break"><h2>For analysis and research</h2>
+<ul><li>R - Statistical computing software: <a href="http://www.r-project.org/">http://www.r-project.org/</a> (Open Source)</li>
+<li>Internet Archive Wayback Machine - retrieve previous verisons of a website: <a href="http://archive.org/web/web.php">http://archive.org/web/web.php</a></li>
+<li>GeoNames - maps names of places to geographic data: <a href="http://www.geonames.org/">http://www.geonames.org/</a></li>
+<li>UN Classifications of the Functions of Government (COFOG): <a href="http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=4">http://unstats.un.org/unsd/cr/registry/regcst.asp?Cl=4</a></li>
+<li>OECD DAR and CRS Code List - used to codify donors and agencies of aid funds: <a href="http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm">http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm</a> </li>
+</ul></div><div Class="objavi-no-page-break"><h2>For visualisation and cleaning </h2>
+<p>Free and Open Source:</p>
+</div><ul><li>Image editing with <em>GNU Image Manipulation Program </em>(GIMP): <a href="http://www.gimp.org/">http://www.gimp.org/</a></li>
+<li>CSVKit - Python library for slicing and dicing csvs: <a href="http://pypi.python.org/pypi/csvkit">http://pypi.python.org/pypi/csvkit</a></li>
+<li>Nominatim - Reverse geocoding: <a href="http://wiki.openstreetmap.org/wiki/Nominatim">http://wiki.openstreetmap.org/wiki/Nominatim</a></li>
+<li>D3 javascript library: <a href="http://d3js.org/">http://d3js.org/</a>,</li>
+<li>Raphael javascript library: <a href="http://raphaeljs.com/">http://raphaeljs.com/</a>,</li>
+<li>Rickshaw javascript library: <a href="http://code.shutterstock.com/rickshaw/">http://code.shutterstock.com/rickshaw/</a></li>
+<li>Natural Language Toolkit (NLTK): <a href="http://nltk.org/">http://nltk.org/</a></li>
+<li>TileMill - quick and beautiful maps: <a href="http://mapbox.com/tilemill/">http://mapbox.com/tilemill/</a></li>
+<li>OKFN recline javascript library for data management: <a href="http://reclinejs.com/docs/tutorials.html">http://reclinejs.com/docs/tutorials.html</a></li>
+<li>Miso javascript library for data management: <a href="http://misoproject.com/">http://misoproject.com/</a></li>
+<li>ScraperWiki - Community scrapers hosted and run on a schedule: <a href="https://scraperwiki.com/">https://scraperwiki.com/</a></li>
+<li>OKFN messy tables: <a href="http://okfnlabs.org/blog/2012/10/22/messytables.html">http://okfnlabs.org/blog/2012/10/22/messytables.html</a></li>
+<li>OKFN BubbleChart: <a href="http://blog.okfn.org/2011/06/29/new-visualisations-for-openspending/">http://blog.okfn.org/2011/06/29/new-visualisations-for-openspending/</a></li>
+</ul><div Class="objavi-no-page-break"><h2>Others</h2>
+<ul><li>Google Fusion Tables - Combine different datasets on a single column: <a href="http://support.google.com/fusiontables/answer/2571232/?hl=en&amp;">http://support.google.com/fusiontables/answer/2571232/?hl=en&amp;</a></li>
+<li>Tumblr - easy blogging platform for quick sharing of visualizations and infographics: <a href="https://www.tumblr.com/">https://www.tumblr.com/</a></li>
+<li>Google Refine : <a href="http://code.google.com/p/google-refine/wiki/DocumentationForUsers">http://code.google.com/p/google-refine/wiki/DocumentationForUsers</a></li>
+<li>Google Charts - Good for HTML/JS developers: <a href="https://developers.google.com/chart/">https://developers.google.com/chart/</a></li>
+</ul></div><h1 id="NetDspqLaA_15"><strong Class="initial">12.</strong> Glossary</h1>
+<div Class="objavi-no-page-break"><h3>aggregated</h3>
+<p>data that is summarized using statistical methods or by simply summing its component parts</p>
+</div><div Class="objavi-no-page-break"><h3>API</h3>
+<p>Application Programming Interface: a specification allowing two pieces of software to interface with each other, without either having knowledge of the inner workings of the other</p>
+</div><div Class="objavi-no-page-break"><h3>backlog</h3>
+<p>a list of tasks prepared by the project manager</p>
+</div><div Class="objavi-no-page-break"><h3>bulk data</h3>
+<p>refers to the entirety of a dataset, instead of the incremental amounts that may be retrieval via other means, such as an API</p>
+</div><div Class="objavi-no-page-break"><h3>CSO</h3>
+<p>Civil Service Organizations: a legally constituted organization created by natural or legal persons that operates independently from any form of government in the interest of civil society. Also commonly referred to as NGOs.</p>
+</div><div Class="objavi-no-page-break"><h3>CSV</h3>
+<p>Comma Separated Values: A format for transactional data that separates each column value by a comma. Can be imported and exported from most spreadsheet programs</p>
+</div><div Class="objavi-no-page-break"><h3>database query language</h3>
+<p>a syntax for interacting with data in a database. Usually specific to the software being used</p>
+</div><div Class="objavi-no-page-break"><h3>hacker</h3>
+<p>one who programs enthusiastically (even obsessively) or who enjoys programming rather than just theorizing about programming (like a 'coder' - but more energetic :) )</p>
+</div><div Class="objavi-no-page-break"><h3>hackday</h3>
+<p>a session where hackers come together to answer a particular task</p>
+</div><div Class="objavi-no-page-break"><h3>iteration</h3>
+<p>a period of a defined length during which changes to a project can be made. Usually ends in a review meeting to plan the next iteration.</p>
+</div><div Class="objavi-no-page-break"><h3>JSON </h3>
+<p>Javascript Object Notation: a data format that is relatively easy for humans to read and write and very easy for machines to read and parse</p>
+</div><div Class="objavi-no-page-break"><h3>Mirroring</h3>
+<p>creating an entire copy of a website on another server, primarily for backup</p>
+</div><div Class="objavi-no-page-break"><h3>MySQL</h3>
+<p>(pronounced "My Sequel" or "MY S-Q-L"): Open source relational database software</p>
+</div><div Class="objavi-no-page-break"><h3>nominatim</h3>
+<p>A web service which turns place names and addresses into coordinates so that they can be mapped / associated with an administrative area</p>
+</div><div Class="objavi-no-page-break"><h3>PostgreSQL</h3>
+<p>(abbreviated Postgres): Open source object relational database software, a query language similar (but not identical) to that in MySQL is used</p>
+</div><div Class="objavi-no-page-break"><h3>OCR</h3>
+<p>Optical Character Recognition: the electronic conversion of scanned images of handwritten, typewritten or printed text into machine-encoded text</p>
+</div><div Class="objavi-no-page-break"><h3>reference data</h3>
+<p>data from outside an organisation (often from standards organisations) which is, apart from occasional revisions, static</p>
+</div><div Class="objavi-no-page-break"><h3>scope creep</h3>
+<p>When a project's scope slowly and steadily increases, without a formal re-examination of the original terms and requirements</p>
+</div><div Class="objavi-no-page-break"><h3>shapefile</h3>
+<p>a file format for geospatial vector data that is primarily used to feed data into mapping applications</p>
+</div><div Class="objavi-no-page-break"><h3>Tilemill</h3>
+<p>Hosted and open source mapping software for creating maps</p>
+</div><div Class="objavi-no-page-break"><h3>user story</h3>
+<p>short descriptions that outline in plain language the different use cases and possibilities for different types of users for the purposes of determining software requirements</p>
+</div><div Class="objavi-no-page-break"><h3>vector data</h3>
+<p>a type of image composed of connected points rather than pixels</p>
+</div><div Class="objavi-no-page-break"><h3>XML</h3>
+<p>eXtensible Markup Language: a markup language created to structure, store, and transport data by defining a set of rules for encoding documents in a format that is both human-readable and machine-readable.</p>
+</div><p>Some of the above entries contain excerpts from Wikipedia.org and the Jargon File v. 4.4.8</p></body></html>

examples/openspending/content/resources/handbook/book-config.js

@@ -0,0 +1,19 @@
+paginationConfig = {
+    'sectionStartMarker': '.objavi-subsection',
+    'sectionTitleMarker': '.objavi-subsection-heading',
+    'chapterStartMarker': 'h1',
+    'chapterTitleMarker': 'h1',
+    'flowElement': "document.getElementById('book')",
+    'alwaysEven': false,
+    'columns': 1,
+    'enableFrontmatter': true,
+    'bulkPagesToAdd': 50, 
+    'pagesToAddIncrementRatio': 1.4, 
+    'frontmatterContents': '<div id="booktitle">Spending Data Handbook</div>'
+	+ '<div id="booksubtitle"></div><div id="bookeditors">'
+	+ 'Iteration 1</div><div class="pagebreak">'
+	+ '</div><div id="copyrightpage">Copyright 2012: Lucy Chambers (OKFN), Lisa Evans (OKFN), Adam Hyde (Booksprints.net), Kaitlin Lee (Sunlight Foundation), Friedrich Lindenberg (OKFN), Damir Mehmedbasic (Public Interest Advocacy Center), Sona Mitra (CBGA), Federico Ramirez (Fundar), Sam Smith (OKFN).<br>License: CC-BY</div>'
+	+ '<div class="pagebreak"></div>',
+    'autoStart': true,
+}
+

examples/openspending/content/resources/handbook/ch001_introduction.html

@@ -0,0 +1,90 @@
+---
+title: Introduction
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Introduction</h2>
+<p>The Spending Data Handbook is addressed to people and organisations who want to use and understand government <strong>budgets</strong> and <strong>spending</strong> data in their work. These groups may include government oversight and accountability groups, issue oriented advocacy organisations, journalists covering the latest round of budget cuts or austerity measures, or citizens who just want to inform themselves on what their government is spending. The truth is that for many countries, it's never been easier to access the underlying data relating to government budgets and spending and the expectations have never been higher for governments to release this information in a timely manner. But once you've got the information, making sense of it can be daunting.</p>
+<p>Irrespective of which kind of user is approaching this data, they all have one thing in common: they need to manipulate, analyse and interpret data in their work<strong><em>.</em> </strong>Increasingly, we are bombarded with statistics and figures from governments and political parties and it is the job of these organisations to scrutinise and fact-check them, as well as come up with alternative models and solutions. This requires timely, data driven analysis.</p>
+<p>If these groups think that <em>"</em>topic-driven analysis<em>"</em> can drive positive change within governments, there is another group at whom this book is addressed: a strange race of people who self-identify as "civic <em>hackers</em>" and think that technology can be put to work to make government as easy, accessible and effortless for citizens as the internet. While these groups use data processing tools in their day to day lives, their focus is often on enabling others to act on information, rather than having a particular advocacy aim themselves (though there are a few notable exceptions to this rule).</p>
+<div id="note">DON'T PANIC! This book uses a bit of technical vocabulary and terms that may be unfamiliar or used differently in different circles. Anything in <em>italics</em> such as <em>hacker</em> in this paragraph, can be found in the glossary at the back, it may not mean what you think.</div>
+<p>The range of topics tackled by these groups is so diverse that it would be impossible to address all of the questions and issues in one humble book. However, there are some overarching principles and helpful techniques which apply universally to working with government financial data and particularly, how technology might help to do so. This book aims to highlight areas where civic hackers, citizens, Civil Society Organisations (henceforth <em>CSO</em>s), and civil servants working in the field of government transparency could combine forces to achieve common aims. Like the Open Data Handbook&#160;(<a title="www.booksprints.net" href="http://opendatahandbook.org/en/">http://opendatahandbook.org/en/</a>), it will be available as a continually evolving, open, educational resource on the internet.</p>
+<h3>What we will cover in this book</h3>
+<ul><li>Collaborating with other organizations to pool resources and strengthen your advocacy effort</li>
+<li>If you're just starting out, what data to look for and what to ask for (nay, demand!) from your government</li>
+<li>The 'Data Pipeline': Tricks and tips for finding, wrangling and systematically processing your data</li>
+<li>Getting ambitious, running a technology project</li>
+<li>Presenting your findings to engage the public, media and government</li>
+<li>Lists and appendices of technical and non-technical resources</li>
+</ul><div id="imagewrapper"><img src="static/pipeline_1.jpg" alt="" width="867" height="505"><p><em>The Data Processing Pipeline</em></p>
+</div>
+<h3>How the book was created</h3>
+<p>The book was started at a four day book sprint bringing together organisations from around the world from both a technical and a CSO background. Representatives from the Open Knowledge Foundation (UK), Fundar (Mexico), the <span>Centre for Budget and Governance Accountability (India), the Public Interest Advocacy Centre (Bosnia and Herzegovina), and the Sunlight Foundation (US) contributed to the original document. The Book Sprint was facilitated by Adam Hyde, founder of the Book Sprint method and <a title="www.booksprints.net" href="http://www.booksprints.net">www.booksprints.net</a>.</span></p>
+<h3>Why are we writing this?</h3>
+<p>As people who work with this data, we know better than anyone that a few years of a handful groups spreading awareness of spending data to the general public has not been sufficient for enacting earth-shattering change in the budgetary policies and processes in governments around the world. While it's true there's more spending data available than ever before, these efforts have proven to be superficial in some cases and the budget processes themselves have not become much more transparent or participatory.</p>
+<p>Conversely, the armchair auditors that were expected to emerge from the citizenry never really materialised. The notion of citizen auditors and engaging with citizens on an individual and collective basis still struggles to make headway. Clearly, we need to take solutions to this problem to the next level. More groups need to be more involved than ever before.</p>
+<h3>What makes this work so tricky?</h3>
+<ul><li>The data people need is not available to the public (and to CSOs).</li>
+<li>Alternatively, the data is not available until after all the important decisions within government have been made.</li>
+<li>It's difficult to simplify data as complex as budgets and spending and make it accessible to a variety of audiences.</li>
+<li>The discourse on budgets and governance in the country is usually replete with jargon and technicalities.</li>
+<li>Even when compelling research findings are presented, there's no <em>sense of urgency</em> on the part of key policy actors. Advocacy with a range of stakeholders becomes important.</li>
+<li>There is duplication of efforts. One CSO may invest two weeks of its resources into painstakingly cleaning up and extracting data from a policy document published as a PDF while another will be doing exactly the same thing.</li>
+<li>Organizations doing this work often lack any kind of peer review process among similar groups.</li>
+<li>There may be skills gaps at many CSOs. Skill and knowledge sharing can help all of them achieve common goals.</li>
+</ul><h3>Contribute to the book</h3>
+<p>This book is released under a Creative Commons attribution licence, meaning that anyone is free to use and reuse the material provided that it is attributed to the Spending Data Handbook. You can help by doing one of the following:</p>
+<ul><li><strong>Translate it and customise it for your region.</strong> The examples which we include here are the ones which we know best, however, you may feel that in your area, there are topics or examples which would be more relevant. Take the book, remix it and add your own examples.</li>
+<li><strong>Correct it and update it </strong>- treat it like a Wikipedia article, a living document. The only way that this book will stay relevant and factually correct is with the help of you and other people who know your stuff.</li>
+<li><strong>Be inspired </strong>- we are filling the book with lots of examples of visualisation and data so inspire you and people you work with.</li>
+</ul></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch002_working-with-others.html

@@ -0,0 +1,64 @@
+---
+title: Working with others
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Working with others</h2>
+<p>We've outlined some ambitious goals for this book, but you don't have to go it alone! In this section we focus on how to work with other organizations to create powerful advocacy efforts, all driven by hard fiscal data! Every CSO, not just government transparency organizations, can leverage the information in this book. Government spending data touches issues covered by a variety of organizations and journalistic outlets, such as human rights, the environment, poverty, scientific research, education, economic policy, and more. But not all these groups may realize it's importance.</p>
+<h3>Why work together?</h3>
+<p>A common observation has been that CSOs, journalists, and other groups that work with a lot of data encounter a set of similar problems. The problems typically include lack of technical knowledge, duplication of work, lack of wide dissemination of analysis and results, and creating compelling visual products. A lot of these challenges could be mollified if proper channels of communication existed between these organizations.</p>
+<p>One of the most important ways to reduce these technical barriers is to maintain an active dialogue and a process of sharing the methodologies and underlying data in analysis you produce. You can avoid the mistakes of your predecessors by reading their methodologies and learning about idiosyncrasies in the data. Or you may be able to point out mistakes another organization has made. Similarly, if you document how your data has been processed and analyzed, other organizations can benefit (more about that in a few chapters).</p>
+<p>Creating communication can be as easy as joining or maintaining a mailing list of groups that work on similar topics or leverage similar data in their work products. Most webmail services offer free group or mailing list functions, such as Google Groups or Yahoo Groups. If you maintain one, be responsive to messages and active in recruiting members. Email is an extremely low barrier for starting a conversation between organizations.</p>
+<p>If you've already got data products that you want to share, try disseminating them in a couple of different ways. In addition to the methods outlined above, try preparing primers or manuals for other groups that want to start working with datasets that you may be particularly experienced with. You can also organize workshops, meetups, and webinars to familiarize groups with the data you work with. If grassroots issue groups can internalize your data products and present them in a contextually relevant way to their user base, then both groups have gained something from the partnership.</p>
+<p>Another good way to facilitate knowledge sharing and communication between organizations is by conducting internship programmes for interested individuals and organisations. However it is important to keep in mind the target groups. It's great to interface with similar spending data focused groups working at different levels of government than your own, but identifying groups that work on completely different issue areas is important. These groups may feel incapacitated to enact change in the fiscal policy as it relates to their own issues. It then becomes an important job for spending data groups to identify and offer their support. It's especially important to establish strong links with groups that serve those minority populations who are often marginalized in government spending priorities.</p>
+<p>The kind of collaboration discussed above can result in a much broader and stronger coalition of organizations that can advocate for fiscal transparency and detailed disclosure of spending data at all levels of government.</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch003_gov-gov-collaboration.html

@@ -0,0 +1,85 @@
+---
+title: Helping the government help itself
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Helping the government help itself</h2>
+<p>We've discussed that it's important for CSOs, journalists, and other groups to work together and it's likely that you won't bump up against too much resistance to this idea. But what about when you get pushback from the government even after you've built a strong coalition to advocate for better access to data? Well, one answer you can give them is "Government darling, you're only hurting yourself".</p>
+<p>Whenever the word 'transparency' is mentioned, the first thought that usually springs to mind is a bright light being shone into dark corners of government offices, trying to expose the dark secrets of corrupt bureaucrats and inefficient expenditure. What is often overlooked is that governments also often stand to benefit from more transparent publishing practices. Not only does proactive disclosure help save on the time and cost associated with increased Freedom of Information requests, but other governments within the same nation can benefit from each other as well.</p>
+<h3>Connecting different levels of government</h3>
+<p>Money at the federal level is often transferred to local governments in the form of grants, direct payments or formula payments. When sub-national governments have better, more timely information on national budgeting priorities, it allows them to adjust their own budgets to account for cuts or increases in certain programs. Similarly, many local governments have small staffs and rely on revenue estimates and models at the federal level to estimate their own revenue.</p>
+<p>This underscores the need for a consistent, standardized information sharing across governments, as it would allow them to share models, best practices, and software with each other, instead of custom building everything from scratch. Furthermore, it would allow for cross-checks on both the federal side and local side for data auditing purposes.</p>
+<p>The next section highlights a couple of case studies for CSOs to use when trying to convince their government to publish better data.</p>
+<h3>Case study</h3>
+<p><em>Transparency to combat lag-times for Governments in British Columbia</em></p>
+<p>When the province of British Columbia built a data portal (<a href="http://www.data.gov.bc.ca/">http://www.data.gov.bc.ca/</a>), its motivations were primarily:</p>
+<ul><li><strong>citizen engagement</strong> - they wanted citizens to better understand the workings of government</li>
+<li><strong>innovation</strong> - they wanted people to build <span>applications and tools using the data</span></li>
+<li><span><strong>making handovers effective</strong> - a large number of the workforce were approaching retirement age, and those in charge wanted to make sure that they handed over the necessary information well in advance</span></li>
+</ul><p>Probably one of the less-expected impacts was seeing how civil servants themselves used the portal. In 2012 approximately one third of all the traffic originated from government computers. The technology enabled faster access to relevant data within the government departments, contributing to better collaboration on policies that required fiscal data. There was also about 20% increase in the number of Freedom of Information (FoI) requests, showing that releasing a small amount of data fuelled wider interest in data.</p>
+<h4>The importance of constant access</h4>
+<p>It is somewhat terrifying to think that a lot of budgeting decisions could made almost entirely in the dark. As soon as one executive budget proposal is finalised and published, work often begins on producing the next one. Within governments, those who have to draw up the next year's plan need access (and quickly) to information, such as actual quarterly expenditures, in order to work out whether a government department is properly resourced from the outset, or is drastically under or over spending.</p>
+<h3>Case study</h3>
+<p><em>International Aid Transparency Initiative (IATI)</em></p>
+<p>Building on around 20 years of previous work, the first version of the International Aid Transparency Initiative (IATI) standard was launched in early 2011 and aid donors around the world started publishing to it. The IATI specification documents data about both aid donors and aid activities, enabling comparison and encouraging good practice in data management. The IATI standard also includes space to attach relevant documents and details of project results, to allow the standard to build context around the data and become an end-to-end solution, tracking projects from inception to execution.</p>
+<p>The IATI standard has seen far greater uptake than any previous effort in terms of organisations electing to publish their data in a standard format. These successes are often attributed to the multi-stakeholder nature of the design process, including both policy makers and technical experts. From 2009 to 2010, IATI consulted with a wide range of stakeholders on the design of the technical standard, alongside a parallel process to secure donor support for publishing their aid information.</p>
+<p>In the past it was common to respond to information shortages by building a new database. But by working with open data principles, IATI allows a more distributed solution - where information can flow between organisations in many different ways, not just into a central database.</p>
+<h4>Main user groups</h4>
+<ol><li><strong>Parliamentarians</strong> in developing countries gain a better oversight of the aid resources available. Knowing where to allocate resources in their own budgeting processes is vital to ensure that money is spent in the best way. Sometimes, there is transparency-asymmetry between different parts of government. For example, a treasury may be very willing to open up the information it holds, but departments which benefit strongly from aid donations (e.g. departments of health) may be more reluctant to be transparent about aid revenues, as they will not want to 'lose out' from central government budgeting.</li>
+<li><strong>Donors</strong>: Know where their money is going and whether it is being spent correctly.</li>
+<li><strong>CSOs</strong> and <strong>private companies</strong> who monitor aid effectiveness</li>
+</ol><p>The benefits of this approach have already been demonstrated, with many CSOs and charities choosing to follow the IATI Standard, although it was developed primarily with governments in mind.</p>
+<p>Donors publish aid information as a feed which can be read by many different applications, including those created by other donors, by the open data community, and - importantly - by software providers that are developing country financial systems. By providing aid information in a standard format, many different users can access the data in the way they need to - and developing countries can see the resources, which are supposed to be flowing to them.</p>
+<h4>Is there anything like this for spending and budgets?</h4>
+<p>At present, no. The way many governments report their financial data is a function of how their budget process works or how they use their accounting software. This may not even be consistent within a government, but vary by department and sub-department.</p>
+<p>Could it be useful to create one? We suspect so. The IATI standard for the first time is enabling people to track the money across country boundaries, something which is almost impossible with other types of expenditure, but crucial for those whose job is oversight of government activities and accountability bodies, such as those who work to prevent international money laundering and corruption.</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch005_introduction-to-data-literacy.html

@@ -0,0 +1,89 @@
+---
+title: Data-driven advocacy and research
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Data-driven advocacy and research</h2>
+<p>We are now in a phase where many governments around the world are proactively publishing documents about what they plan to spend (budgets) and actually spend (spending data). Increasingly, this material is available on the internet, so that anybody can access it at any time. Still, too much of the information is released in the form of '<em>documents</em>' rather than '<em>data</em>'. Ideally we need both so that inforamtion can be analyzed, re-used and understood. This chapter is a quick overview of some of the raw inputs required for data-driven advocacy and how it works in practice.&#160;</p>
+<h3>What do we mean by machine-readable data?</h3>
+<p>When we speak about data, what we usually refer to is the notion of machine-readable&#160;(<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://en.wikipedia.org/wiki/Machine-readable_data">http://en.wikipedia.org/wiki/Machine-readable_data</a>) data. Many of the formats most commonly used for policy papers and long-form reports published by most policy-making institutions are PDF files, Word documents, web pages or closed interactive infographics - do not structure information in a way that lends itself to automated analysis and extraction.</p>
+<p>Such documents are formatted for humans (or printers) to interpret, and it can be hard (and in many cases nearly impossible) for a machine to re-construct the elements in the presentation.</p>
+<p>Other formats, such as Excel and CSV files contain a higher level of structured information. For example, in an Excel file you can mark a number of cells and easily calculate their sum. Even more exotic and useful file formats, such as&#160;<em>XML documents</em>, <em>JSON API</em>s or <em>Shapefiles</em> may not have easy-to-use viewer applications. You can think of them as the glue that connects different systems on the web, so that different databases can work together in a seamless fashion.</p>
+<h4>Why do CSOs need it?</h4>
+<p>What asking for machine-readable bulk data means for CSOs is simple: you won't have to spend a lot of time manually extracting data from reports into spreadsheets to be able to filter, sort and analyse it - a process which is both time-consuming and can introduce errors.</p>
+<h3>What to ask for when asking for data: a checklist</h3>
+<p>In the next section 'Getting Data' - we will deal with asking governments for data (or getting it via other means). To set the scene for this and to work out whether your government actually publishes usable data already, have a quick look at the following questions:</p>
+<ul><li><strong>Is the government's data published in a machine-readable format?</strong> E.g. CSV, XML, JSON. While there is nothing wrong with publishing a PDF to support a data release (in fact it is often nice to have a nicely-laid out document to cross reference and sanity-check data) it shouldn't be the only thing published and if you are asking for a policy document, ask for the underlying data in a spreadsheet so you can check the numbers.</li>
+<li><strong>Does the government publish a '<em>data dictionary</em>' to explain the terms used in the dataset?</strong> This should include definitions of column headers, explanations of terms and ranges used within the main body of the data, explanations of any changes in terminology which have been introduced since last time the dataset was released</li>
+<li><strong>How is the data that is being published <em>actually</em> used internally by governments?</strong> Do some sanity checks on the minimum and maximum values of different columns to make sure they fall into the documented ranges and don't seem out of place. Do you see negative values when you don't think you should? Negative values usually mean money owed.</li>
+<li><strong>Is the structure of the data the same across years? If not is there a description of how it changes?</strong> It never hurts to contact the publisher and ask questions about the change and why it occurred. The publisher may have their name and contact details on the report or webpage. If there is no named contact then call the department's enquires number or send a message to their email address asking to meet or discuss the data.</li>
+<li><strong>How aggregated is the data?</strong> What is the number of real-world financial transactions that are expressed by a single line of the dataset you have? For budgets this will mostly be hard to tell - but with transactional expenditure you want to make sure that the data is fairly disaggregated. Ideally, each entry represents a transaction - but even if this isn't true you'll still want to ensure the number is not in the tens or hundreds of thousands (e.g. government programmes as a whole).</li>
+<li><strong>Ask for reference data.</strong> If your budget or spending data is augmented with reference data, make sure you have access to it. This might include functional or category codes on budget line items, location codes for describing recipient location, or codes that indicate the status of the record.&#160;</li>
+<li><strong>Ask also for the guidelines people were given when creating the dataset.</strong> This will make it easier to understand what is included within the data, e.g. are the numbers in thousands / millions.&#160;</li>
+<li><strong>Final tip: if the data you want is not given then narrow your scope.</strong> Your chances of success will be higher if you narrow the scope of the data you're requesting from the government and you are specific. Government is the de facto keeper of all kinds of data, so parameters that narrow your request are always helpful.</li>
+</ul><h3>An introduction to data-driven advocacy</h3>
+<p class="p1">Is going out and provoking a riot the best way to get a Government to take onboard your message? There are alternatives: hit them with the data hammer instead!</p>
+<p class="p1">Making evidence-based policy proposals consists from three major phases: formulating your assumption, analysis (which often leads to re-formulating your assumption, and presenting your data in an engaging way in a policy proposal.</p>
+<h4 class="p1"><strong>Analysing assumptions</strong></h4>
+<p class="p1">Asking the right question is key to getting the most out of your data. We all make assumptions, and our organisation may have a particular standpoint on a given issue. Our first task is always to formulate our assumptions and then interrogate them ferociously. Although we try to be rational in this process, our judgement is often influenced by our subjective goals, values, and beliefs.&#160;Sometimes, you'll need to revisit your assumptions several times over to ensure they are valid and you can back them up with data. Once you know your policy problem is definitely a problem, you can work to package it in a way that's appropriate for your target audience.&#160;</p>
+<h4 class="p1">What is public interest?&#160;</h4>
+<p class="p1">Often our job is to act in the public interest by analysing conflicting assumptions and working out which one is more valid. For example, in Greece, Spain, and many other European countries people protest almost everyday as the Government cuts spending to bring down its budget deficit. If the Goverment wanted to keep its current level of spending, but increased taxes to increase its revenue, different citizens groups would still protest depending on which taxes are to be increased. In any case, there will always be more than one interpretation of any Government policy, and interested side to support it, or not.</p>
+<h4 class="p1"><strong>Policy analysis</strong></h4>
+<p class="p1">Once we have a well defined policy problem, specific goals, or results different stakeholders are trying to achieve, and corresponding instruments they are using in this process, we may systematically search for the specific data needed to create our own policy proposals. This data can be obtained either from the Government, some other sources e.g. academic journals, private companies, or generated by ourselves. When data is gathered we will use a specific methodology to analyze it, and based on this analysis we will approve or reject our assumptions. If the assumption is rejected, based on our findings we will have to make the new assumption, and start the process from the beginning. If our assumption is approved, we will use our results to make a policy proposal to the Government.</p>
+<h4 class="p1"><strong>Policy proposals</strong></h4>
+<p class="p1">For CSOs it is important to recognize who is a decision maker, hence, who you should be targeting with your policy proposal.&#160;Policy proposals should be methodologically well structured, evidence-based, open for debate, and scientificaly evaluated. Governments will seldom take our policy proposals as their own policymaking, but may actually change its course of action, get new insights, views, and understanding of the subject. We may also use policy briefs to approach Government officials, or press releases to get the attention of the public.&#160;</p>
+<p class="p1"><img src="static/cso-3.jpg" alt=""></p>
+<h3 class="p1"><strong>Case study&#160;</strong></h3>
+<p class="p1"><em>Fish subsidies</em></p>
+<p class="p1">The influence CSOs have on government policy comes from a wide and varied set of activities. These can range from producing a widely shared dataset or infographic which subtly influences the mood of policy makers, to more targeted CSO&#160;advocacy&#160;and lobbying on issues they are experts.</p>
+<p class="p1">The Fish Subsidies group (<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://fishsubsidy.org">http://fishsubsidy.org</a>) are a nice example of a CSO engaged in targeted activites. Having collected a comprehensive set of data on Fishing subsidies&#160;paid under the European Union&#8217;s common fisheries policy and they break this down into payments for&#160;every EU member state, and then&#160;complemented this with&#160;activites of fishing. They have produced a report (<a title="http://en.wikipedia.org/wiki/Machine-readable_data" href="http://is.gd/XYPgq5">http://is.gd/XYPgq5</a>) assessing the environmental and social impacts of the Financial Instrument for&#160;Fisheries Guidance between 2000 and 2006. This extensive document fed directly into the EU political decision making process. &#160;</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch006_types-of-data.html

@@ -0,0 +1,91 @@
+---
+title: Types of data
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Types of data</h2>
+<p>For the purposes of this manual, we have decided to abstract the meaning of the terms "budget data" and "expenditure data" to fit into a broader context. While these terms may have different meanings on a country by country basis, they are intended to be taken as defined in this section, throughout the rest of the manual. In this section, we look briefly at the two different types of data and what questions can be addressed using them.</p>
+<p><img src="static/pipeline-ask-a-question.jpg" alt="" width="856" height="445"></p>
+<p><strong>Budget data</strong> is defined as data relating to the broad funding priorities set forth by a government, often highly <em>aggregated</em> or grouped by goals at a particular agency or ministry. For instance, a government may pass a budget which contains elements such as "Allocate $20 million in funding for clean energy grants" or "Allocate $5 billion for space exploration on Mars". These data are often produced by a parliament or legislature, on an annual or semi-annual basis.</p>
+<p><strong>Spending data</strong> is defined as data relating to the specific expenditure of funds from the government. This may take the form of a contract, loan, refundable tax credit, pension fund payments, or payments from other retirement assistance programs and government medical insurance programs. In the context of our previous examples, spending data examples might be a $5,000 grant to Johnson's Wind Farm for providing renewable wind energy, or a contract for $750,000 to Boeing to build Mars rover component parts. Spending data is often transactional in nature, specifying a recipient, amount, and funding agency or ministry. Sometimes, when the payments are to individuals or there are privacy concerns, the data are aggregated by geographic location or fiscal year.</p>
+<p>The fiscal data of some governments may blur the lines of these definitions, but the aim is to separate the <em>political documents</em> from the <em>raw output of government activity</em>. It will always be an ultimate goal to link these two datasets, and to allow the public to see if the funding priorities set by one part of the government are being carried out by another part, but this is often impractical in larger governments since definitions of programs and goals can be "fuzzy" and vary from year to year.</p>
+<p><img src="static/budget-spending-data-3.jpg" alt="" width="874" height="1240"></p>
+<h3>Budget data</h3>
+<p>Using the definitions above, budget data is often comprised of two main portions: <em>revenue</em> and <em>taxation data</em> and <em>planned expenditures</em>. Revenue and spending are two sides of the same coin and thus deserve to be jointly considered when budget data is released by a government. Especially since revenue tends to be aggregated to protect the privacy of individual taxpayers, it makes more sense to view it alongside the budget data. It often appears aggregated by income bracket (for personal taxes) or by industrial classification (for corporate taxes) but does not appear at all in spending data. Therefore, budget data ends up being the only source for determining trends and changes in revenue data.</p>
+<p>Somewhat non-intuitively, revenue data itself can include expenditures as well. When a particular entity or economic behaviour would normally be taxed but an exception is written into the law, this is often referred to as a <em>tax expenditure</em>. Tax expenditures are often reported separately from the budget, often in different documents or at a different time. This often stems from the fact that they are released by separate bodies, such as executive agencies or ministries that are responsible for taxation, instead of the legislature<span style="font-size: 12px;">&#160;</span>(<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://internationalbudget.org/wp-content/uploads/Looking-Beyond-the-Budget-2-Tax-Expenditures.pdf">http://internationalbudget.org/wp-content/uploads/Looking-Beyond-the-Budget-2-Tax-Expenditures.pdf</a>).</p>
+<h4>Budgets as datasets</h4>
+<p>A growing number of governments make their budget expenditure data available as <em>machine-readable</em> spreadsheets. This is the preferred method for many users, as it is accessible and requires few software skills to get started. Other countries release longer reports that discuss budget priorities as a narrative. Some countries do something in between where they release reports that contain tables, but that are published in PDF and other formats from which the data is difficult to extract.</p>
+<p>On the revenue side, the picture is considerably bleaker, as many governments are still entrenched in the mindset of releasing revenue estimates as large reports that are mostly narrative with little easily extractable data. Tax expenditure reports often suffer from these same problems.</p>
+<p>Still, some areas that relate to government revenue are beginning to be much better documented and databases are beginning to be established. This includes budget support through development aid, for which data is published under the IATI (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://www.aidtransparency.net/">http://www.aidtransparency.net/</a>) and OECD DAC CRS (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://stats.oecd.org/Index.aspx?DatasetCode=CRSNEW">http://stats.oecd.org/Index.aspx?DatasetCode=CRSNEW</a>) schemes. Data about revenues from extractive industries is starting to be covered under the EITI (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://eiti.org/">http://eiti.org/</a>) with the US and various other regions introducing new rules for mandatory and granular disclosure of extractives revenue. Data regarding loans and debt is fairly scattered, with the World Bank providing a positive example (<a title="http://www.publicdebtnet.org/public/Statistics/" href="https://finances.worldbank.org/">https://finances.worldbank.org/</a>), while other major lenders (such as the IMF) only report highly aggregated figures. An overview of related data sources can be found at the Public Debt Management Network (<a title="http://www.publicdebtnet.org/public/Statistics/" href="http://www.publicdebtnet.org/public/Statistics/">http://www.publicdebtnet.org/public/Statistics/</a>).</p>
+<h4>Connecting revenues and spending</h4>
+<p>It is highly desirable to be able to determine the flow of money from revenues to spending. For the most part, many taxes go into a general fund and many expenditures come out of that general fund, making this comparison moot. But in some cases, in many countries, there are taxes on certain behaviours that are used to fund specific items.</p>
+<p>For example, a car registration fee might be used to fund the construction of roads and highways. This would be an example of a user fee, where the main users of the government service are funding it directly. Or you might have a tax on cigarettes and alcohol that funds healthcare grants. In this case, the tax is being used to offset the added healthcare expense of individuals taking part in at-risk activities. Allowing citizens to view what activities are taxed in order to pay for other expenditures makes it possible to see when a particular activity is being cross-subsidized or heavily funded by non-beneficiaries. It can also allow them to see when funds are being diverted or misused. This may not always be practical at the country level, as federal governments tend to make much larger use of the general fund than other local governments. Typically, local governments are more comprehensive with regards to releasing budget data by fund. Having granular, fund-level data is what makes this kind of comparison and oversight possible.</p>
+<p><img src="static/funding_1.jpg" alt="" width="867" height="614"></p>
+<h4>What questions can be answered using budget data?</h4>
+<p>Budget expenditure data has an array of different applications, but it's prime role is to communicate to it's user broad trends and priorities in government spending. While it can help to have a prose accompaniment, the data itself promotes a more clear-cut interpretation of proposed government spending over political rhetoric. Additionally, it is much easier to communicate budget priorities by economic sector or category than it is at the spending data level. These data also help citizens and CSOs track government spending year over year, provided that the classification of the budget expenditure data stays relatively consistent.</p>
+<h3>Spending data</h3>
+<p>For most purposes, spending data can be interpreted as transactional or near-transactional data. Rather than communicating the broad spending priorities of the government like budget data should, spending data is intended to convey specific recipients, geographic locations of spending, more detailed categorization, or even spending by account number.</p>
+<p>Spending data is often created at the executive level, as opposed to legislative, and should be more frequently reported than budget data. It can include many different types of expenditures, such as contracts, grants, loan payments, direct payments for income assistance and maintenance, pension payments, employee salaries and benefits, intergovernmental transfers, insurance payments, and more.</p>
+<p>Some types of spending data - such as contracts and grants - can be connected to related procurement information (such as the tender documents and contracts) to add more context regarding the individual payments and to get a clearer picture of the goods and services covered under these transactions.</p>
+<h4>Opening the checkbook</h4>
+<p>In the past five years, there have been a spate of countries and local governments that have opened up spending data, often referred to as "checkbook level" data. These countries include, but are not limited to, the US (including various state governments), UK, Brazil, India (including some state governments) and many funds of the European Union.</p>
+<h4>Disclosure thresholds</h4>
+<p>At least two of these countries have imposed seemingly arbitrary thresholds on the size of transactions that are included. For example, the US and the UK exclude transactions under $25,000 and 25,000 GBP, respectively. Are these thresholds appropriate? That can't be known for sure without more information about how these numbers were arrived at. Principally, having thresholds or exceptions to the reporting of this data depends on the underlying systems that drive disclosure of this data. Are these systems linked directly with the accounting systems already used in the government, easing the burden of disclosure? If so, the threshold for excluding transactions should be very low (setting aside for a moment the cases that require redaction for privacy purposes).</p>
+<p>If the systems are mostly divorced, as is the case with the US, then it begs the question, why? The more steps and processes between the internal government accounting systems and the public accounting systems, the higher the chance of error in the data and chance for omission of data. It also undermines the primary goal of public oversight if there are separate systems. However, governments often struggle with IT resources and contracting, which presents a tension between releasing any spending data at all, and release that is consistent with the above principles. If a threshold is necessary, then the amount should be consistent in size and scope with the overall expenditure level for that particular government. It is not appropriate, for example, that the threshold for the US State of Maryland spending reporting is also $25,000, when their annual budget is only a fraction of the federal government's budget.</p>
+<h4>Release early, release often</h4>
+<p>Spending data should be released in a relatively timely fashion, at least a monthly or quarterly basis. The timeliness of this data is what allows users to see if the spending priorities in the budget data are being reflected in the spending data. Also it allows the public and government stakeholders to view the current year's spending on a more detailed level as the next year's budget is being decided.</p>
+<p>A good example of such release of spending information can be the Indian experience, especially the <span>Employment Guarantee Programme, one of the major National flagship programmes on providing demand-based employment to the rural working age-group population in India. Its Management Information system (MIS) has become the most effective way of getting information on spending on a monthly basis. The data is updated monthly in an accessible spreadsheet format (Excel) at the sub-national government level. This makes the data transparent and available in the public domain to be equally accessed by all. The village level household database has internal checks for ensuring consistency and conformity to normative processes. It includes separate pages for approximately 250,000 local governments at the village level, 6,465 Blocks, 619 Districts and 34 States &amp; Union Territories. The portal places complete transaction level data in the public domain.</span></p>
+<p><span>However, problems related to maintaining an MIS in every state in a functional form and releasing continuous flow of data have been contentious issues. The major concerns emerge from lack in technical capacity as well as cost related issues. A</span> cumbersome back-end system for supplying the data requires installation of a specific software with prerequisite configurations and technical operators with specific capacities. These requirements have raised costs and put a great demand on technology to ensure a continuous flow of data on the programme, specifically in most interior parts of the country and hence affect timely release of data.</p>
+<h4>What questions can be answered using spending data?</h4>
+<p>Spending data can be used in several different areas: oversight and accountability, strategic resource deployment by local governments and charities, and economic research. However, it is first and foremost a primary right of citizens to view detailed information about how their tax dollars are spent. Tracking who gets the money and how it's used is how citizens can detect preferential treatment to certain recipients that may be illegal, or if certain political districts might be getting more than their fair share.</p>
+<p>It can also help local governments and charities respond to areas of social need without duplicating federal spending that is already occurring in a certain district or going to a particular organization. Lastly, businesses can see where the government is making infrastructure improvements and investments and use that criteria when selecting future sites of business locations. These are only a few examples of the potential uses of spending data. It's no coincidence that it has ended up in a variety of commercial and non-commercial software products -- it has a real, economic value as well as an intangible value as a societal good and anti-corruption measure.</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch007_getting-cleaning.html

@@ -0,0 +1,140 @@
+---
+title: Getting and cleaning data
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Getting and cleaning data</h2>
+<p>As you look into a new, data-driven project, you'll find it easy to imagine all the analysis and comparisons that could be done to prove a particular point or to explore some area of government activity. As you then look into the availability of data on a particular topic, disillusionment will quickly set in: information may be hard to find, unstructured or just not available to the public at all. As you search for data, there are a number of possible techniques you may want to investigate. Some are based on the governments cooperation while others use technology to bridge the gap between how information is shared and what is necessary for a data-driven advocacy project.</p>
+<h3>Getting data</h3>
+<div id="imagewrapper"><img src="static/pipeline-getting-data.jpg" alt="" width="874" height="501"> &lt;p &gt;<em>Data processing pipeline</em></div>
+<p>An increasing number of governments have accepted the need for pro-active transparency. When releasing open data, governments commit to making machine-readable information available on the web, in a form that enables anyone to use, re-use and re-distribute the data without legal or technical restrictions. Such releases should happen in a timely manner and include detailed, disaggregated data. Many countries have set up dedicated open data portals which provide easy access to all published datasets and relevant metadata (i.e. information about when a dataset was last updated, who published it and where documentation regarding format and contents can be found).&#160;The benefit of using open data when it's available includes the ease of access, but also usually a more authoritative base for any further analysis.</p>
+<p>In some cases, open data is provided through an application programming interface (API), often a web-based method for retrieving, searching or even updating the available information dynamically. APIs provide up-to-date data in a granular and filtered form, removing the need to repeatedly process and update source files.</p>
+<p>A common use case for APIs is relatively time-sensitive information, such as procurement calls and contracts which are released every day. In the UK, BusinessLink (<a title="http://opencorporates.com" href="http://www.contractsfinder.businesslink.gov.uk/data-feed.aspx">http://www.contractsfinder.businesslink.gov.uk/data-feed.aspx</a>) provides a number of data feeds which contain information about procurement notices. Similarly, the USASpending portal provides a set of APIs that can be used to retrieve up-to-date grants information for the US federal government (<a title="http://opencorporates.com" href="http://usaspending.gov/data">http://usaspending.gov/data</a>).</p>
+<div id="note">If you are looking for a list of data catalogs from around the world, DataCatalogs.org is curated by experts in this area from around the world. Search for your country and see what you find.</div>
+<div id="lightimage"><img src="http://farm8.staticflickr.com/7127/7631434818_d8f903252c_z.jpg" alt="" width="640" height="397"></div>
+<h4>Freedom of information</h4>
+<p>Even before the rise of open data, many countries decided to increase the transparency of their governments by introducing freedom of information (FoI) legislation. Such laws enable every citizen to request documents and other material from parts of the government which do not merit special protection (e.g. due to concerns over privacy, national security or commercial confidentiality).</p>
+<p>Journalists, activists and CSOs have long had channels of acquiring information. Sometimes, having a good relationship with a press-officer or a civil servant is good enough and making a formal request for information is unnecessary (your friendly press-officer may even feel slightly offended if you don't ask them nicely first). FoIs generate a lot of paperwork (hence grumpy civil servants), so if you do have the contacts, it may be a good idea to ask nicely first!</p>
+<p>Freedom of Information requests often require some degree of preparation, so that the documents or databases that are requested are clearly identified, you know which department or unit is in charge of it and you can address possible concerns over privacy or commercial confidentiality in your request.</p>
+<p>While freedom of information legislation is in force in many countries, it was often made before the need for structured data became apparent - thus many laws do not allow the citizen to specify a particular format. Many governments choose to release information on paper rather than in a structured, digital form, making the data processing step more painful. Still, the legally binding character of freedom of information requests often makes them an invaluable tool in the process of gaining access to financial data.</p>
+<p>FoI requests may be necessary when you want to get more detail on the projects that government money is funding. Often the transactional spending data released will include only a brief description of the project, if at all. To get more information about it, you might need to submit an FoI request. For instance, if you have the high level payment information for a contract that includes the recipient, location and total amount, but you want to know the details of the contract deliverables, you will probably need to submit an FoI request for the full contract.</p>
+<p>A good example of this process is the Sunlight Foundation's request for information on the Airport Improvement Program in the United States. The program accepts applications from airports around the country for infrastructure improvement grants, such as repaving a runway. Each project is assigned a safety priority rating and is prioritized in a queue. The high level spending information for this program was available in USASpending.gov, but since the priority ratings are specific to this program and not spending data in general, they were not included in that dataset. The Sunlight Foundation submitted a FoI request for the full dataset, including the priority ratings. After that, they were able to determine when airports with low priority projects were getting money, and how often. So the lesson is, if you see some interesting patterns in your high level spending data, don't be afraid to dig deeper and ask for more detailed program information.</p>
+<p>Wanting to submit a request, but not sure where to start, who to address your request to or how to write it? Access Info (<a title="http://opencorporates.com" href="http://www.access-info.org/">http://www.access-info.org/</a>) are an organisation who work to help people obtain the information they require from the public bodies that hold it. They have also produced a toolkit (<a title="http://opencorporates.com" href="http://www.legalleaks.info/toolkit.html">http://www.legalleaks.info/toolkit.html</a>) to using FoIs. It's primarily aimed at Journalists, but most of the tips are equally relevant for CSOs.</p>
+<h4>Data scraping</h4>
+<p>Unlike open data or freedom of information requests, data scraping does not rely on the cooperation of government authorities for acquiring machine-readable documents. Scraping refers to the process of transforming unstructured documents - online database interfaces, PDF files or even printed documents into a more structured form (see section below on Optical Character Recognition for more tips on how to do this). Many tools and techniques enable such refinement by re-interpreting documents generated for humans into a more structured form which can be analysed, aggregated and distributed in new ways.</p>
+<p>While there is an increasing number of easy-to-use scraping tools which do not require much technical knowledge, more complex processes - such as the automated scraping of thousands or millions of web sites or the mass interpretation of PDF files require some programming so that the process can be fully automated and produce reliable results.</p>
+<p>In some cases, the only way to gain access to a set of figures is through the digitization of printed material. While scanners and optical character recognition (OCR) software can be used to import such documents, the high cost and low data quality generated through this approach often it an unattractive one.</p>
+<p>A very accessible guide to scraping has been published (<a title="http://opencorporates.com" href="https://leanpub.com/scrapingforjournalists">https://leanpub.com/scrapingforjournalists</a>). Again, it brands itself at journalists (they're getting a little spoiled aren't they?), but scraping is scraping and so a worthwhile read for CSOs.</p>
+<p>Another great source for tutorials, as well as being a tool itself to help with scraping is ScraperWiki. Using Scraperwiki (<a title="http://opencorporates.com" href="https://scraperwiki.com/">https://scraperwiki.com/</a>) has the additional benefit of anything that you scrape being available to others, as most of the scraped data goes into a public data store.</p>
+<p>&#160;</p>
+<h4>Getting data out of scanned documents</h4>
+<p>When you deal with scanned documents, the crucial step in the extraction process is to have the computer attempt to recognize any characters - letters, numbers and other signs. Optical character recognition (OCR) software is built to do this, accepting scanned pictures and PDF documents as an input.</p>
+<p>There are both commercial software products for OCR (such as ABBYY FineReader, <a title="http://opencorporates.com" href="http://finereader.abbyy.com/">http://finereader.abbyy.com/</a>), and some open-source software packages, such as Google's Tesseract (<a title="http://opencorporates.com" href="http://code.google.com/p/tesseract-ocr/">http://code.google.com/p/tesseract-ocr/</a>). In general, the quality of all automatic recognition is limited, and you should make sure to cross check any numbers coming from scanned material against the printed documents.</p>
+<h4>Keeping the data around</h4>
+<p>As you retrieve data from the government (or other sources), it's easy to just consider the websites it has been released on as a permanent resource. Still, experience has shown that data does go away: whether it is through government re-designing its web sites, new policies that retract transparency rules or simple system failures.</p>
+<p>At the same time, downloading complete copies of web sites - a process called <em>mirroring</em> - is a fairly well-established technique that can easily be deployed by civil society organisations. Mirroring involves an automated computer program (for a list see: <a title="http://opencorporates.com" href="http://en.wikipedia.org/wiki/Web_crawler">http://en.wikipedia.org/wiki/Web_crawler</a>) harvesting all the web pages from a specified web page, e.g. a ministry home page. In most cases, it is also possible to find old versions of web sites via the Internet Archive's Wayback machine (<a title="http://opencorporates.com" href="http://archive.org/web/web.php">http://archive.org/web/web.php</a>), a project that aims to create up-to-date copies of all public web sites and archive them forever.</p>
+<h3>Cleaning data</h3>
+<div id="imagewrapper"><img src="static/pipeline-cleaning-data.jpg" alt="" width="874" height="486"></div>
+<p>&lt;p &gt;<em>Data Processing Pipeline</em></p>
+<p>As you acquire government spending information, you will notice that such data often has many inconsistencies: program names are used inconsistently, financial amounts will be stated in badly formatted numbers, while some data may not be usable at all due to file corruptions. In short: data always needs to be cleaned and processed. In fact, processing, augmenting and cleaning the data is very likely to be the most time- and labour-intensive aspect of your project.</p>
+<h4>Processing stages for data projects</h4>
+<p>While there are many different types of data, almost all processing can be expressed as a set of incremental stages. The most common stages include data acquisition, extraction, cleaning, transformation, integration, analysis and presentation. Of course, with many smaller projects, not each of these stages may be necessary.</p>
+<p>In this process, each stage fulfils a unique purpose</p>
+<ul><li><strong>Acquisition</strong> describes gaining access to data, either through any of the methods mentioned above or by generating fresh data, e.g through a survey or observations.</li>
+<li>In the <strong>extraction</strong> stage, data is converted from whatever input format has been acquired (e.g. XLS files, PDFs or even plain text documents) into a form that can be used for further processing and analysis. This often involves loading data into a database system, such as <em>MySQL </em>or <em>PostgreSQL</em>.</li>
+<li><strong>Cleaning and transforming the data</strong> often involves removing invalid records and translating all the columns to use a sane set of values. You may also combine two different datasets into a single table, remove duplicate entries or apply any number of other normalizations.</li>
+<li><strong>Analysis</strong> of data to answer particular questions we will not describe in detail in the following chapters of this book. We presume that you are already the experts in working with your data and using e.g. economic models to answer your questions. The aspects of analysis which we do hope to cover here are automated and large-scale analysis, showing tips and tricks for getting and using data, and having a machine do a lot of the work, for example: network analysis or natural language processing.</li>
+<li><strong>Presentation</strong> data only has impact when it is packaged in an appropriate way for the audiences it needs to aim at.</li>
+</ul><p>As you model a data pipeline, it is important to take care that each step is well documented, granular and - if at all possible - automated. This is particularly true when processing more complex datasets, such as transactional government expenditure which has been scraped from a government website.</p>
+<h3>Data provenance</h3>
+<p>Good documentation on data provenance (the origin and history of a dataset) can be compared to the chain of custody which is maintained for criminal investigations: each previous owner of a dataset must be identified, and they are held accountable for the processing and cleaning operations they have performed on the data. For Excel spreadsheets this would include writing down the steps taken in transforming the data, while advanced data tools (such as Open Refine, formerly Google Refine), often provide methods of exporting machine-readable data containing processing history. Any programs that have been written to process the data should be available when users access your end result and shared as open-source code on a public code sharing site such as GitHub.</p>
+<h4>Tools for documenting your data work</h4>
+<p>Documenting the transformations you perform on your data can be as simple as a detailed prose explanation and a series of spreadsheets that represent key, intermediate steps. But there are also a few products out there that are specifically geared towards helping you do this. Socrata is one platform that helps you perform transforms on spreadsheet-like data and share them with others easily. You can also use the Data Hub (pictured below), an open source platform that allows for several versions of a spreadsheet to be collected together into one dataset, and also auto-generates an API to boot.</p>
+<div id="lightimage"><img src="static/Screen%20Shot%202012-11-15%20at%2015.35.38.png" alt="" width="947" height="585"></div>
+<h4>Normalizing data</h4>
+<p>Data that comes from the government is often generated across multiple departments by hand. This can result in inconsistencies in what kinds of values or formats are used to describe the same meaning. Normalizing values to be consistent across a dataset is therefore a common activity.</p>
+<p>First, you want to start by finding all of the distinct ranges of values for the different columns in your dataset. You can accomplish this by using a database query language (such as SQL's DISTINCT), or by simply using the 'filter' property on a spreadsheet program.</p>
+<p>For example, if you have a spreadsheet with contracting data, and one column is 'Competed?', you would expect the values to be 'yes' or 'no'. But if this spreadsheet is an amalgam of spreadsheet data from multiple users and departments, your values could vary among the following: 'Y', 'YES', 'yes', 1, 'True', 'T', 't', 'N', 'NO', 'no', 0, 'False', 'F', 'f', etc. Limiting all of these potential values to two clear options will make it easier to analyse the data, and also easier for those who follow in your footsteps.</p>
+<p>Especially with financial data, numbers can be formatted several different ways. For example, are your negative values represented with a '-' or placed inside '( )' or possibly even highlighted in red? Not all of these values will be easily read by a computer program (especially the color), so you'll want to pick something clear and consistent to convert all your negative values to (probably the negative sign).</p>
+<p>Is all your numerical data measured out in ones or is abbreviated in thousands? Especially with budget data, order of magnitude errors are not uncommon when one department thinks they're reporting in thousands or millions by default but others expand their data all the way to the ones place. Are some values in scientific notation (e.g. 10e3938)? Make sure all your values are consistent, otherwise your analysis could contain serious errors.</p>
+<div id="lightimage"><img src="static/Screen%20Shot%202012-11-15%20at%2011.34.49%20AM.png" alt="" width="297" height="373"><em>A column of data requiring name normalization</em></div>
+<h3>Data quality</h3>
+<p>Data cleaning and data quality go hand in hand. You may find it helpful to conduct data quality tests after your analysis in order to measure your confidence in the data, or before to decide whether you want to do the analysis at all. Data quality can mean a few different things but here or some key factors to consider.</p>
+<h4>Completeness</h4>
+<p>Is all your data there? If you have a time series of some data, do you have approximately the same number of rows for each year or are there major outliers? An extremely high or low number of rows that looks out of place may indicate duplicate or lost rows. Or it could indicate a change in how data was reported at that time. Are there values for each column where values are required (according to the documentation)?</p>
+<h4>Accuracy</h4>
+<p>Does your dataset match up with other available data out there? This can be difficult to confirm, but it is often useful to employ a sampling methodology. Select a significant sample of your records (3-5%) and decide which values you can check for accuracy. If your data is transactional contract data, you could request the full contracts for your sample records and compare the structured data you have with the prose of the contract to confirm that it is correct. You can sometimes check contract and grant amounts from journalistic outlets and even company websites.</p>
+<h4>Timeliness</h4>
+<p>Sometimes governments can lag behind in their reporting. Even if a dataset says it's for a specific time period, do you notice a long tail of records closer to the end of the time frame? If so, you may want to restrict your analysis to the subset of data where you have a healthy sampling.</p>
+<h4>Case studies</h4>
+<p>Some projects have gone beyond simple assessments of data quality to ensure confidence in the underlying result of a project, but are projects completely about data quality themselves!</p>
+<p><em>The Open Knowledge Foundation recently created a data quality monitor for UK transactional expenditure data (<a title="http://opencorporates.com" href="http://openspending.org/resources/gb-spending">http://openspending.org/resources/gb-spending</a>) which has been developed in cooperation with the data.gov.uk team. Each imported file is documented, highlighting any mistakes in the data structure or formal representation of the data. More information at:</em></p>
+<p><em>The Sunlight Foundation has run a yearly report called Clearspending (<a title="http://opencorporates.com" href="http://sunlightfoundation.com/clearspending/">http://sunlightfoundation.com/clearspending/</a>) for the past three years that assesses the quality of the data reported in the US federal spending dashboard, USASpending.gov. The results haven't been very good but have highlighted the need for data standardization and oversight of the data release process.</em></p>
+<p>As an even more thorough technique, little programs and tests can be written to check data validity along the processing pipeline by testing certain assumptions about each record in the dataset. If you do not have the option of creating such tests, you may still be able to create a spreadsheet in which you record known defects and issues with the data, including source files that did not open and inconsistent column values.</p>
+<p>The most powerful data quality tool is much more simple, though: sharing the data. Without access to your source data, others will have to trust your word blindly. Therefore, sharing both the source and processed forms of your dataset is essential to discussing data quality.</p>
+<h3>Categorization and reference data</h3>
+<p>One of the most powerful ways of making data more meaningful for analysis is to combine it with <em>reference data</em> and <em>code sheets</em>. Unlike transaction data - such as statistical time series or budget figures - reference data does not describe observations about reality - it merely contains additional details on category schemes, government programmes, persons, companies or geographies mentioned in the data.</p>
+<p>For example, in the German federal budget, each line item is identified through an eleven-digit code. This code includes three-digit identifiers for the functional and economic purpose of the allocation. By extending the budget data with the titles and descriptions of each economic and functional taxonomy entry, two additional dimensions become available that enable queries such as the overall pension commitments of the government, or the sum of all programmes with defence functions.</p>
+<p>The main groups of reference data that are used with government finance include code sheets, geographic identifiers and identifiers for companies and other organizations:</p>
+<h4>Classification reference data</h4>
+<p>Reference data are dictionaries for the categorizations included in a financial datasets. They may include descriptions of government programmes, economic, functional or institutional classification schemes, charts of account and many other types of schemes used to classify and allocate expenditure.</p>
+<p>Some such schemes are also standardized beyond individual countries, such as the UN's classification of functions of government (COFOG) and the OECD DAC Sector codes&#160;(<a title="http://opencorporates.com" href="http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm">http://www.oecd.org/dac/aidstatistics/dacandcrscodelists.htm</a>). Still, the large majority of governments use their own code sheets to allocate and classify expenditure. In such cases, it is often advisable to request access to the code list versions used internally by government, including revisions over time that may have changed how certain programmes were classified.</p>
+<p>A library of reference data that can be re-used across different projects and it is a valuable asset for any organization working with government finance. Sharing such data with others is crucial, as it will help to enable comparable outputs and open up options for future data integration. Existing repositories include the IATI Standard (<a title="http://opencorporates.com" href="http://iatistandard.org/">http://iatistandard.org/</a>) and datahub.io.</p>
+<h4>Geographic identifiers</h4>
+<p>Geographic identifiers are used to describe administrative boundaries or specific locations identified in a dataset. While some regional classifications (such as the EU NUTS) are released on the web, there is also an increasing number of open databases which contain geographic names - including&#160;geonames.org&#160;and the recently developed world.db.</p>
+<p>Another related technique is the process of reverse geo-coding: translating a human-readable address into a pair of coordinates. Services like&#160;nominatim&#160;(<a title="http://opencorporates.com" href="http://nominatim.openstreetmap.org/">http://nominatim.openstreetmap.org/</a>)&#160;will not only enable users to generate precise maps of projects in a region, they will also return the responsible administrative boundary for many coordinates. This means that projects which are given by precise address can also be aggregated by state, region or any other geographic unit.</p>
+<p>Additionally, many countries have shapefiles of their political and geographic districts available (usually through the census or interior bureaus) that can be imported into custom mapping applications, like TileMill&#160;(<a title="http://opencorporates.com" href="http://mapbox.com/tilemill/">http://mapbox.com/tilemill/</a>).</p>
+<h4>Company and organisational identifiers</h4>
+<p>As you look into spending data that includes recipients outside the government, you'll find companies which act as suppliers to government, but also other types of entities including charities, associations, foreign governments, political parties and even individuals which act as recipients of direct assistance.</p>
+<p>Identifying such entities is notoriously hard, since the only information kept by government is often a simple name (which may not uniquely identify the beneficiary, for example "MS"). While most (if not all) countries maintain company registers which assign some type of unique identifier to a company, these databases are often not accessible in bulk and not used coherently across different parts of government. Alternative identifiers - such as tax identifiers and company IDs from private business information suppliers (such as Dun &amp; Bradstreet in the US) - further complicate this process.</p>
+<p>As an alternative, open registries are beginning to compile organisational identifiers in a form that is easy to re-use and thus enables the sharing of databases which have been augmented with such information. OpenCorporates.com (<a title="http://opencorporates.com" href="http://opencorporates.com">http://opencorporates.com</a>) is a startup that collects information from companies world-wide and provides a convenient API to match datasets with the list of known countries. The IATI project for aid transparency is working towards similar standards for other organisations, such as foreign governments and charities active in the development space.</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch008_anaysis.html

@@ -0,0 +1,75 @@
+---
+title: Analysis
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Analysis</h2>
+<p style="text-align: center;"><img src="static/pipeline-analysing-data.jpg" alt="" width="874" height="475"></p>
+<p style="text-align: center;"><em>Data Processing Pipeline</em></p>
+<p style="text-align: center;">&#160;</p>
+<div>Dear reader - we know you are the experts. We're not going to teach you all the basics, but this section's aim is to show you how technology can help speed up, scale up or simplify your working practices. For this reason in this section we will say only a few words about statistical methods, even though many of you will be very familiar with those.</div>
+<p>Once you have your data in a suitable format to be dealt with, much of your analysis can be automated with a custom program or inside spreadsheet software. Obviously the deeper your data dive is, the less automated things become, but having a standard set of post-processing, analytical steps to run on a dataset helps make sure you haven't missed anything. Some of the steps may seem basic, but it always helps to have a checklist!</p>
+<p><strong>Sorting data</strong></p>
+<p>Possibly the simplest way to find some interesting points is to sort by a column or category. In spending datasets, it's common to rank data by the largest transactions and look at the beneficiaries. This also gets you the maximum or minimum values of a column, which could be interesting.</p>
+<p><strong>Basic statistical analysis</strong></p>
+<p>Statistical analysis such as calculating percentiles or the average of a dataset, are essential to characterize your data. These are available in most programming languages as well as most spreadsheet programs like excel. A more complete list of statistical tools and resources can be found at the resources chapter.</p>
+<ul><li>An <strong>average</strong> is a measure of central tendency in a sample. It can be described as the "typical" or representative value of a data set. Knowing the average helps detect whether any single payment is atypical.</li>
+<li>A <strong>percentile</strong> is the value of a variable under which a certain percentage of the data fall. For example, if a transaction amount is in the 90th percentile, then it is larger than 89% of the transactions in the dataset.</li>
+<li>A <strong>standard deviation</strong> measures how far away a value is from the average. If your standard deviation is low, then it means your row values are clustered together. If it's high, then they are spread out. By checking for values that are one, two, or three standard deviations outside the mean, you can detect outliers. Outliers could be errors or extremely interesting data points.</li>
+</ul><p><strong>Summarizing or aggregating by categories</strong></p>
+<p><strong></strong>Sometimes data is available at a very high level of detail, making it hard to see the big picture. It's a good idea to have a standard set of categories to aggregate by, such as geographic unit, functional classification, account number, fiscal year or other temporal unit, government department, or even recipient.</p>
+<p><strong>Normalization for comparability</strong></p>
+<p><strong></strong>In order to be able to compare data across time or other parameters, you may need to systematically adjust the data values. Here are some examples.</p>
+<ul><li><strong>Deflation or inflation</strong>. Expenditure data is naturally obtained with current (or nominal) money value, but such data is not directly comparable across different years since inflation affects the value of money. In order to compare currency across years, it is necessary to deflate or inflate the value according to a base year. For example if we wish to compare the budget of a program in dollars from 2000 with the same program in 2012 we must adjust the value of the 2000 data and the 2012 dollars to a base year. Deflator and inflator factors are calculated by central banks in most countries and a comprehensive list by country can be found at the World Bank website (<a title="http://worldbank.org" href="http://worldbank.org">http://worldbank.org</a>)</li>
+<li><strong>Scale and normalize values</strong>. When we wish to compare patterns of budget programs sometimes the difference in scale between budgets may prove difficult to display in a graphic form. In such cases is useful to compare percentages instead of absolute numbers. For instance, it is very common to express budget and spending data as percentages of a country's GDP, instead of in absolute numbers. If you're mapping spending, make sure to scale your data by population density. Otherwise, your map will most likely just be a map of where your populace lives.</li>
+</ul><div id="note">You may want to visualize your data to get better understanding of your initial assumptions. This may save&#160;you a lot of time in case your initial assumptions were not good, and may also help you to get more insight into new findings your data might contain but previously were not clearly visible. For this purpose you can use scatter plots, line graphs, heat maps, bar charts.</div></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch009_ngo-ngo-collaboration.html

@@ -0,0 +1,91 @@
+---
+title: Using technology in your work
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Using technology in your work</h2>
+<p>As you move into more advanced data projects, you may find yourself in a situation where a research project turns into an effort that involves coders, designing databases and web sites. It is important that you take a step back and realize that you are now running not just an advocacy effort, but also an IT project.</p>
+<p>There are many difficulties which CSOs face when developing software. Some common issues include:</p>
+<ul><li>Difficulties in finding qualified developers that want to contribute to your projects at a reasonable rate, as well as in the communication between CSO staff and developers.</li>
+<li>Clearly communicating the requirements for the software so that both non-technical and technical staff share a vision for the outcome of the project.</li>
+<li>The estimation of time and resources for particular tasks, especially how to handle projects that drastically overrun the timeframe and funding they were initially assigned.</li>
+<li>Evaluating the work of developers to ensure that the product that has been delivered is according to what has been agreed, especially in small projects with only a single coder or when working with external contractors.</li>
+<li>Maintaining the project after the main development period has finished</li>
+</ul><h4>Starting from scratch vs. re-using Components</h4>
+<p>It is always going to be more costly and riskier to develop something from scratch than to customise something that already exists.</p>
+<p>You should make the software behind your projects open source. If many other organizations do the same, this allows code to be reused across jurisdictions. Not only does this ease the financial burden, but it helps create the expectation in populaces around the globe for the high quality engagement tools that their neighbouring country has access to.</p>
+<p>That's not to say you should never develop something new - just ask around first, and make sure that what you are asking is technically feasible.</p>
+<h4>Commissioning New Software</h4>
+<p>Purchasing software is more closely related to having a piece of clothing made than to buying chairs. You can give the designer a basic vision of what you would like, but you will always need to come back to make sure it really fits, and your thoughts may change when you see things in practice. If you have an arrangement with your tailor which allows you to first specify the general idea, and a couple of other appointments for fittings and trials, you'll probably end up with a better and more creative result than if you tried to design the whole thing and it was simply unveiled to you at the end. You'll also feel more in control and it may even be quicker to do design and implementation in parallel.</p>
+<h4>Defining requirements &amp; Designing the solution</h4>
+<p>Define the basic components of your project and prioritise them by their importance. As the developers start working on one of these chunks, you can then break it up into more specific tasks based on your evolved understanding of the project. A popular technique for this purpose are <em>user stories</em>, small narrative pieces that describe each problem: "As a [web site visitor] I want to [be able to see a supplier's contracts] so that I can [understand what services they provided to government]". The key to these stories is that they describe the actual user need, not the details of the solutions that you have envisaged. While you should of course discuss those with the developers as well, defining solutions is mainly the job of the developers, not the project manager.</p>
+<h4>Implementation: iterations and milestones</h4>
+<p>A saying amongst developers goes: "Walking on water and developing software from a specification are easy if both are frozen." As your software project is progressing you will likely realize that the specifications you have given need to be revised or extended. Yet by modifying the requirements you are essentially shifting the ground on which developers are executing - meaning they will have to stop their work to adjust. To prevent such changes from freezing all development, the process of introducing changes and additional requirements needs to be structured.</p>
+<p><em>Iterations</em> are periods of a defined length - often two or four weeks - during which developers are tasked to execute a set of previously selected user stories or requirements. Before the iteration starts, developers have to pull in the work from a list of tasks (a so-called <em>backlog</em>) prepared by the project manager, committing themselves to delivering those tasks within the agreed period. Crucially, project managers are not allowed to extend or revise the scope of an iteration while it is ongoing (unless they want to declare it failed). This method ensures that changes are introduced in bulk and understood by the team. This approach mandates the opposite of the more common unstructured communication between managers and developers, e.g. emails with unsorted lists of change requests which tend to be ignored and lead to confusion.</p>
+<p>Whenever you consider an additional requirement, be sure to consider if it is realistic within the resources you have available. "Scope creep", the progressive extension of a project during its development, is a common cause of project failure. By becoming more and more ambitious, the project finally ends up with no usable product at all. To avoid scope creep, make sure to have a storage area for long-term ideas. Also make sure that developers accept additional tasks through a pull process, and not by having them pushed into their workflow.</p>
+<h4>Maintenance</h4>
+<p>Make sure to budget a for ongoing maintenance after the end of your project. Who is going to guarantee that the servers stay online? Who is going to fix a typo? It is unlikely that your project will remain entirely static after its initial development, so you should have an explicit agreement with the developers regarding future support. It is also useful to collect feedback after the projects release to commission a small number of additional days when enough additional work has accumulated.</p>
+<h3>Roles and how to find developers</h3>
+<p>The key ingredient to a successful software development project is having the right people on staff or as contractors. Depending on the scope and type of your project, you may need a variety of skills - these are some of the common descriptions:</p>
+<ul><li><strong>Web designers</strong> typically produce designs and layouts for web pages, often initially in a graphics program like Adobe Photoshop. Most, but not all, web designers then translate their designs into web markup (HTML, CSS).</li>
+<li><strong>Web developers</strong>&#160;are more technical. They produce interactive web interfaces such as search masks, browsers or specific form-based operations. They often use programming languages such as PHP, Ruby on Rails, Python or JavaScript.</li>
+<li><strong>Visualization designers</strong> develop graphics that represent quantitative information. A key distinction here is between non-interactive graphics (i.e. static images) and interactive visualizations, which often require some programming. There are still very few designers who design interactive visualizations, so rates may be relatively high.</li>
+<li><strong>Software developers</strong> are even more technical, developing backend software for data processing or acquisition. They are experienced in the use of database software (such as SQL databases) and programming languages such as Ruby, Python or Java.</li>
+<li><strong>Data scientists</strong> and <strong>statisticians</strong> produce analysis based on large sets of data, detecting tendencies and outliers in the dataset. They are not usually expected to produce front-end applications, but may produce software in the process of analysing data.</li>
+<li><strong>Usability experts</strong> and <strong>user experience (UX) designers</strong> think about the way your user will interact with your site answering questions such as 'is it obvious from the landing page what the purpose of this site is?'</li>
+<li><strong>Testers</strong> try and break things to test their robustness. This is particularly useful e.g. if you think your project will receive a lot of traffic as a result of a media campaign, you want to know your site can survive the hit</li>
+</ul><h4>Good places to look for developers</h4>
+<p>The easiest way to meet developers is through community meetups, such as <em>hackdays</em>. During such events, coders meet up to cooperatively develop prototypes of new software. To meet volunteer developers who can help you make sense and unleash the power of government spending and budgets, it's wortwhile to investigate events such as Random Hacks of Kindness (<a title="http://techcampglobal.org/" href="http://www.rhok.org/">http://www.rhok.org</a>),&#160;Data Kind (<a title="http://techcampglobal.org/" href="http://datakind.org/">http://datakind.org/</a>) and TechCamps (<a title="http://techcampglobal.org/" href="http://techcampglobal.org/">http://techcampglobal.org/</a>).</p>
+<p>There are a few ways you can discover if there is a hackday in your area. One way is to search on Lanyrd (<a title="http://techcampglobal.org/" href="http://lanyrd.com/search/?q=hackday&amp;context=future">http://lanyrd.com/search/?q=hackday&amp;context=future</a>) or set up an account on that system and request that you are alerted when there is a hackday in your area.&#160;Another approach is join mailing lists of organisations that might help you find developers e.g. the Open Knowledge Foundation lists (<a title="http://techcampglobal.org/" href="http://lists.okfn.org/mailman/listinfo">http://lists.okfn.org/mailman/listinfo</a>) or the Sunlight Labs mailing list (<a href="https://groups.google.com/forum/#!forum/sunlightlabs">https://groups.google.com/forum/#!forum/sunlightlabs</a>).</p>
+<h4>What do things cost?</h4>
+<p>It's impossible to give concrete guidelines on how much a project should cost. Developers' salaries are generally quite high for a country's average, but vary very strongly from country to country. Worried about your project spiraling out of control? We'd recommend agreeing on a price per iteration, and it may be a good idea to draw up a contract which allows you to break it off if you are not happy with the work at the end of an iteration. Plus, you can generally also find a friendly developer to glance over a quote from a company for a sanity check.</p></div>
+    </div>
+</div>

examples/openspending/content/resources/handbook/ch011_defining-the-scopetopic.html

@@ -0,0 +1,98 @@
+---
+title: Presentation and engagement
+layout: handbook
+---
+
+<div class="row">
+    <div class="span4">
+        <ul class="nav nav-list span3">
+        
+            <li class="nav-header">Introduction</li>
+            
+                <li><a href="ch001_introduction.html">Introduction</a></li>
+            
+                <li><a href="ch002_working-with-others.html">Working with others</a></li>
+            
+                <li><a href="ch003_gov-gov-collaboration.html">Helping the government help itself</a></li>
+            
+        
+            <li class="nav-header">Data Literacy</li>
+            
+                <li><a href="ch005_introduction-to-data-literacy.html">Data-driven advocacy and research</a></li>
+            
+                <li><a href="ch006_types-of-data.html">Types of data</a></li>
+            
+                <li><a href="ch007_getting-cleaning.html">Getting & cleaning data</a></li>
+            
+                <li><a href="ch008_anaysis.html">Analysis</a></li>
+            
+                <li><a href="ch009_ngo-ngo-collaboration.html">Using technology in your work</a></li>
+            
+        
+            <li class="nav-header">Presentation and engagement</li>
+            
+                <li><a href="ch011_defining-the-scopetopic.html">Presentation and engagement</a></li>
+            
+                <li><a href="ch012_selecting-methods-and-tools.html">Selecting methods and tools</a></li>
+            
+        
+            <li class="nav-header">Appendices</li>
+            
+                <li><a href="ch014_resources.html">Resources</a></li>
+            
+                <li><a href="ch015_glossary.html">Glossary</a></li>
+            
+        
+            <li class="nav-header">Further information</li>
+            <li><a href="http://okfn.booktype.pro/spending-data-handbook/">Contribute to the book</a></li>
+            <li><a href="spending-data-handbook.pdf"><strong>Download a PDF version</strong></a></li>
+            <li><a href="Spending_Data_Handbook.epub">ePub Version (iPad)</a></li>
+            <li><a href="Spending_Data_Handbook.mobi">MOBI Version (Amazon Kindle)</a></li>
+        </ul>
+    </div>
+    <div class="span8">
+        <div><h2>Presentation and engagement</h2>
+<p>While raw financial data is valuable just as a tool for research, analysis and investigation, it can also be directly presented to an audience in a wide variety of ways. Such presentations can take the form of press releases in which data is used to make an argument, interactive infographics which condense information to visually represent a key finding or even searchable on-line databases that allow each user to find spending in their neighbourhood.</p>
+<p><img src="static/pipeline-presenting-data.jpg" alt="" width="874" height="456"></p>
+<h3>Clarifying your topic</h3>
+<p><em>"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away" - Antoine de Saint Exup&#233;ry</em></p>
+<p>If you are reading this, you probably already have a fairly clear idea of the topic that you want to investigate and present. Still, it is useful to look at different ways of defining your subject. In some cases, you may merely want to give an overview of a particular area, e.g. by showing the proportions of government spending which go into the various functions of government. In other cases you may have a particular hypothesis that you want to portray or a particular journalistic story that you mean to tell, such as a case of systemic corruption that becomes evident only as you contrast one department's spending patterns with that of another one.</p>
+<p>Sometimes there is a temptation to get lost in complexity. Using all the data and tools available to you to their full potential can force an almost endless amount of information onto our audiences. The benefit of doing so is limited, however: people will not understand the point that you are making, or, even worse, ignore you.</p>
+<p>At the same time, technology can also be used to do the opposite: focus in on the key points, hiding other detail. Selecting where to go into depth doesn't have to be the author's responsibility any longer, it can be a choice that the user of data-driven applications is empowered to make based on their own interests. The prerequisite for using technology to create a clear focus is answering the question: What is the core of what we mean to present, what is the essence of the narrative our project is intended to convey?</p>
+<h3>Setting an objective</h3>
+<p>As you focus the topical core of your project, you also need to think about the objectives of your project: is it intended to inform the general public about an aspect of government budgeting or spending practices? Are you making a specific case to government, e.g. by proposing a new policy which you have gathered evidence and developed arguments for?</p>
+<p>Your project can also serve as a forum for discussions, which would usually require you to facilitate the debate to maintain ongoing engagement.</p>
+<p>The objective may be to establish a network-based, collaborative effort, e.g. to build a community that may review documents released by the government or evaluate the conditions of public contracts. Such modes of engagement require extensive design to encourage users to contribute in a way which best matches their particular skills and interests.</p>
+<p>Some example of working with the public to collaboratively clean up financial data is Linked Gov (<a title="http://yourbudgit.com" href="http://linkedgov.org/">http://linkedgov.org/</a>). This is a platform that will allow the public and civil servants to access and do small tasks that will clean government financial data.</p>
+<h3>Targeting an audience</h3>
+<p>The efficiency of projects based on spending data depends heavily on how well you have defined and understood your audience. If your message is too general for a specialized audience, it will be ignored. At the same time, a complicated and very technical presentation will usually fail to address broader parts of the population. Whether it be that your main audience is a specific constituency, government officials, the general public or a combination of those, it is important that the media and methods you apply help to get the message(s) across to the audience that you are targeting.</p>
+<h4><strong>The general public</strong></h4>
+<p><strong></strong>Is the purpose of your campaign to stimulate discussion amongst the general public? Unless your organization has exceedingly well-developed outreach channels, you will not usually be able to address the general public directly. Still, thanks to the social media and internet-based direct communication, many organizations can now contact and involve citizens who have shown some interest in their work directly. In cases where your main audience is the general public, simplicity is a must. It is necessary that the vast majority of people get a grasp of what you are trying to communicate. In these cases you may need to emphasize the visual presentation of the data that gets quickly and succinctly to the point, such as infographics. See the case study at the end of this chapter for how an organisation in Nigeria used Social Media to reach out to the general public.</p>
+<h4><strong>Mass media</strong></h4>
+<p><strong></strong>Unlike direct channels, broadcast and print media require you to first convince an intermediary of the relevance and quality of your message. Media are often the best channels to use when we want to create social pressure on governments to change some budget issue.</p>
+<h4>Community and constituency</h4>
+<p><strong></strong> This may include people and organizations with which you have previously cooperated or which are active in the same domain. Unlike the wider public, they are more likely to be interested not just in your conclusions, but also in the data and tools that you have used to reach your conclusions. Creating an open space for your community to discuss your work is essential in getting relevant feedback. When you need to reach to your constituency it may be necessary to provide a broader context, such as a full report including methodology, conclusions and possible paths of further discussion.</p>
+<h4>Government</h4>
+<p>Officials are often a target of your outreach, since they are in the position of actually deciding and implementing changes in policy. While they can be reached through public pressure and the media, your organization or some of its constituency may also choose to contact governments directly. In the case of trying to reach government officials, a balance of the previous approaches is very important. Meetings with government officials are rare and short-lived. Condense your main argument and supporting evidence into one page or less, and include a clear action that you want them to take.</p>
+<p>Obviously, on any given issue you may choose to involve any set of these groups, but it is still important to have a clear picture of who they are and what their incentives motivate them.</p>
+<p><img src="static/pyramid_1.jpg" alt="" width="870" height="678"></p>
+<h3>Case study</h3>
+<p><em>BudgIT Nigeria on social media</em></p>
+<p>Data is invaluable. To the uninformed person where information asymmetry thrives, access to data illuminates the path to facts and provokes emotions that trigger results. Nevertheless, poor handling of data puts valuable facts in an opaque structure that communicates nothing. Data could be in a maze of thick document riddled with complex terms or iterations of figures that doesn&#8217;t connect the user. Not being able to promote discussion nor give provide contextual understanding, data may could be worthless.</p>
+<div id="lightimage"><img src="static/nigeria-budget2.png" alt="" width="826" height="728"></div>
+<p>Nigeria returned to democracy in 1999 after lengthy years of military rule. Data under the barrel of the gun was an exclusive preserve of rulers as probing the facts behind data is taken as an affront to authority and strive to question the stained reputation of the junta. Civil servants were bound by the Official Secrets Act not to share government information hereby putting the citizens in the dark. Even after thirteen years of return to democracy, there is clearly a gap in accessing public data with some government officials still stricken with military era hangover. Data especially in terms of public expenditure communicates little to the larger sections of public who are not versed in financial accounting nor have requisite understanding to evaluate the complex arithmetic.</p>
+<p>BudgIT, creative start-up sees a huge opportunity in using creative data visualization to stimulate interests concerning public expenditure. Understanding the ubiquity of the mobile device within the Nigerian locality and the increasing number of Nigerians online, BudgIT sees the opportunity to engage Nigerians and explain public expenditure in a simpler way. This thrives on building engagement across all platforms, encouraging the community to action via NGOs and reaching out to everyone to promote citizen participation. This is about making public data a social object and building an extensive network that demands change.</p>
+<h4>Building The Community</h4>
+<p>Across our interactions with users, we see a gulf in understanding what the budget is and what the citizens usually expect. We have engaged over 10,000 Nigerians over the budget and we profile them into three to ensure optimum value is delivered. The categories are briefly explained below:</p>
+<p>Singular Users: These are users who want information in simple and quick format. There are interested in data but not in a deeper scale of analyzing the content or probing further. A series of simple tweets is enough for them to retell the stories or interactive applications that give a snapshot.</p>
+<p>Closed Loop Users: A set of users who stimulate a discussion pattern, engage the data channel effectively to increase their knowledge of subject matter or challenge the assumptions of data. These set of users are adherents of the platform via social media, hence they ensure in building the platform through feedback systems or referral to their social connections.</p>
+<p>The Data Hog: This set of users want the raw datasets to rework visualization or do extensive analysis for personal or enterprise purposes. We simply give them the data for their definitive purposes.</p>
+<h4>Engaging the Citizen</h4>
+<p>Across every society lies a literacy span and engaging every component is highly critical to the societal growth and stability. There is always a storyboard in every life that volumes of data can be matched with. People constantly want to be more informed especially concerning issues that they find difficult to understand. Engaging citizens is to take critical analysis of the target users at a time and itemize the possible profiles. A look at user profile demands a thorough analysis of their empathy, attention and insight towards the data available to them. What does the Nigerian citizen care about? Where is the information gap? How quickly can we reach out to them and place data in the storyboard of their lives? A critical understanding of the user&#8217;s psychology and the perceived response to the data is the first needed level of analysis. BudgIT&#8217;s immediate reach is to the average literate Nigerian connected to online forums and social media. Most online users amidst the array of interests in gaming, reading and sharing social connections within a limited timeframe will definitely need data in a brief and concise manner. After a snapshot of data either as a tweet or infographics, there&#8217;s an opportunity to build linked data on other platforms where the big picture can be set and interaction can be enhanced. &lt;p &gt;An important angle of visualization to us is understanding the data appreciation level of the users. Complex diagrams, superb infographics and aesthetic interactive applications might not convey the exact meaning to user based on his/her previous approach to data. Data vizualization needs to take into consideration how users can easily grasp the vizualized data and subject it to personal interpretation. A good visualization transfers knowledge and mostly important brings forth a story the user can easily connect with.</p>
+<p>For us in BudgIT, our engagement model is anchored on the following:</p>
+<ul><li>Stimulating discussion around Trends: In engaging with users on public data, BudgIT keeps track of online and offline discussions and seeks to provide data to enliven the interactions. A glaring example was the fuel strikes in January 2012 where there was a constant agitation among the protesters on the need to reduce the size of governance. BudgIT tracking the discussion via social media in 36 active hours quickly built an app that allows citizens to cut the Nigerian budget. The huge response with over 3,000 users who interacted with the budget using the app refined our engagement model. We keep looking for trends in the polity and matching it with relevant data quickly rendered into tweets or infographic display that quickly &#160;extends our influence.</li>
+<li>Constructive feedback mechanism and Balanced outlook: Data speaks volumes and individuals subject it to personal interpretations. In the engagement with users, feedback is enabled through discussion boards or retweets. Most users throw up discussions that tend to ask about stories behind the data and seeking opinions of BudgIT. Its of utmost priority to ensure that opinions only explain the facts behind the data and does not conform to individual disposition to the subject matter. It&#8217;s most necessary to build up feedback channels and engage the users creatively to ensure the community built around the data is sustained.</li>
+<li>Localize Outlook of Data: For a dataset targeted at a particular group, BudgIT is building competency to localize its content and promote a channel of discussion that connects to the users. This involves taking a cultural outlook at the icons, symbols, objects and language to ensure that engagement concerning the budget flows seamlessly. This phase of engagement is with the grassroots who mostly don&#8217;t have access to Internet but only possess SMS-based mobile phones.</li>
+</ul><p>After making the public expenditure data available in an easy-to-read format as shown on our portal (<a title="http://yourbudgit.com" href="http://yourbudgit.com">http://yourbudgit.com</a>), we reach out to the citizens through the civil society organizations to ensure that citizens monitor capital projects in the budget. We also plan to develop a participatory framework where citizens and government institutions can meet in town halls to define key items in the budget that needs to be prioritized. Once we get citizens to be aware of capital projects in the budget and connect them with civil societies where BudgIT is not located, citizens can track report projects and report status. Ensuring that citizens of any literacy span are armed with data and possess a clear path to demand action, BudgIT is crossing the rubicon from open data to open action.</p></div>
+    </div>
+</div>