THIS REPOSITORY IS NO LONGER MAINTAINED BY THE EDIPLOMACY TEAM. IT IS NOW THE RESPONSIBILITY OF THE FOREIGNASSISTANCE.GOV TEAM. CONTACT THE FOREIGNASSISTANCE.GOV TEAM IF INTERESTED IN THIS CODEBASE.
This is an online tool that enables users to explore and analyze national level data, and then share what they discover. This was designed around the needs of U.S. government analysts and also benefits a broad-range of stakeholders, including the U.S. public, civil society organizations, universities, foreign aid donors, Congress and partner country governments.
In development environment, run inside of client-dev-app/. This is a separate application and for production deployments can be injected into the final application build. (dust template script includes to bundle.js)
#How to run the code in development
- Install Postgres 9.4
- Restore from the latest backup for development. Contact [email protected] for the backup.
- Validate there were no errors with the restore
- Under app,
npm install - Create a app/config/config.json file from app/config/config.sample.json . You can do this from the top-level directory of the repo with
cp app/config/config.sample.json app/config/config.json. - Add your development database credentials to the config.json file you just created
- Under client-dev-app,
npm install - Compile custom plotly.js icons into a custom plotly.js bundle
cd into node_modules/plotly.js/
Run
npm iRunnpm run build - Run
npm run buildin /client-dev-app. This will create a client-dev-app/build directory. - Copy the client-dev-app/build directory to app/.build . Note that the directory under app is a hidden directory. You can do this from the top-level directory of the repo with
cp -r client-dev-app/build app/.build. - While in the top-level directory, run the server
node app/server.js
- Folder Structure
- Available Scripts
- Installing a Dependency
- Importing a Component
- Adding Stylesheets with SASS
- Adding Images and Fonts
- Using the
publicFolder - Adding Custom Environment Variables
- Integrating with a Node Backend
- Proxying API Requests in Development
- Running Tests
- Filename Conventions
- Command Line Interface
- Version Control Integration
- Writing Tests
- Testing Components
- Using Third Party Assertion Libraries
- Initializing Test Environment
- Focusing and Excluding Tests
- Coverage Reporting
- Continuous Integration
- Disabling jsdom
- Experimental Snapshot Testing
- Editor Integration
- Deployment
- Troubleshooting
Project structure should look like this:
find-fa-visualize/
README.md
node_modules/
package.json
public/
index.html
favicon.ico
... bundled/transpiled filed here (production)
src/
actions/
... redux actions
components/
... react components
containers/
... react container components (parent level)
reducers/
... redux state management reducers
index.js
sass/
... application sass (broken up by utils (scss variables), containers, and components for modularity)
styles/
... outputted scss (do not edit these files, use the sass directoy)
store.js
index.js
For the project to build, these files must exist with exact filenames:
public/index.htmlis the page template;src/index.jsis the JavaScript entry point.
You can delete or rename the other files.
You may create subdirectories inside src. For faster rebuilds, only files inside src are processed by Webpack.
You need to put any JS and CSS files inside src, or Webpack won’t see them.
Only files inside public can be used from public/index.html.
Read instructions below for using assets from JavaScript and HTML.
You can, however, create more top-level directories.
They will not be included in the production build so you can use them for things like documentation.
In the project directory, you can run:
Runs the app in the development mode.
Open http://localhost:3000 to view it in the browser.
The page will reload if you make edits.
You will also see any lint errors in the console.
Launches the test runner in the interactive watch mode.
See the section about running tests for more information.
Builds the app for production to the build folder.
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.
Your app is ready to be deployed!
See the section about deployment for more information.
Note: this is a one-way operation. Once you eject, you can’t go back!
If you aren’t satisfied with the build tool and configuration choices, you can eject at any time. This command will remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (Webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except eject will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
You don’t have to ever use eject. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
The generated project includes React and ReactDOM as dependencies. It also includes a set of scripts used by Create React App as a development dependency. You may install other dependencies (for example, React Router) with npm:
npm install --save <library-name>
This project setup supports ES6 modules thanks to Babel.
While you can still use require() and module.exports, we encourage you to use import and export instead.
For example:
import React, { Component } from 'react';
class Button extends Component {
render() {
// ...
}
}
export default Button; // Don’t forget to use export default!import React, { Component } from 'react';
import Button from './Button'; // Import a component from another file
class DangerButton extends Component {
render() {
return <Button color="red" />;
}
}
export default DangerButton;Be aware of the difference between default and named exports. It is a common source of mistakes.
We suggest that you stick to using default imports and exports when a module only exports a single thing (for example, a component). That’s what you get when you use export default Button and import Button from './Button'.
Named exports are useful for utility modules that export several functions. A module may have at most one default export and as many named exports as you like.
Learn more about ES6 modules:
With Webpack, using static assets like images and fonts works similarly to CSS.
You can import an image right in a JavaScript module. This tells Webpack to include that image in the bundle. Unlike CSS imports, importing an image or a font gives you a string value. This value is the final image path you can reference in your code.
Here is an example:
import React from 'react';
import logo from './logo.png'; // Tell Webpack this JS file uses this image
console.log(logo); // /logo.84287d09.png
function Header() {
// Import result is the URL of your image
return <img src={logo} alt="Logo" />;
}
export default Header;This ensures that when the project is built, Webpack will correctly move the images into the build folder, and provide us with correct paths.
This works in CSS too:
.Logo {
background-image: url(./logo.png);
}Webpack finds all relative module references in CSS (they start with ./) and replaces them with the final paths from the compiled bundle. If you make a typo or accidentally delete an important file, you will see a compilation error, just like when you import a non-existent JavaScript module. The final filenames in the compiled bundle are generated by Webpack from content hashes. If the file content changes in the future, Webpack will give it a different name in production so you don’t need to worry about long-term caching of assets.
Please be advised that this is also a custom feature of Webpack.
It is not required for React but many people enjoy it (and React Native uses a similar mechanism for images).
An alternative way of handling static assets is described in the next section.
Note: this feature is available with
[email protected]and higher.
Normally we encourage you to import assets in JavaScript files as described above. This mechanism provides a number of benefits:
- Scripts and stylesheets get minified and bundled together to avoid extra network requests.
- Missing files cause compilation errors instead of 404 errors for your users.
- Result filenames include content hashes so you don’t need to worry about browsers caching their old versions.
However there is an escape hatch that you can use to add an asset outside of the module system.
If you put a file into the public folder, it will not be processed by Webpack. Instead it will be copied into the build folder untouched. To reference assets in the public folder, you need to use a special variable called PUBLIC_URL.
Inside index.html, you can use it like this:
<link rel="shortcut icon" href="%PUBLIC_URL%/favicon.ico">Only files inside the public folder will be accessible by %PUBLIC_URL% prefix. If you need to use a file from src or node_modules, you’ll have to copy it there to explicitly specify your intention to make this file a part of the build.
When you run npm run build, Create React App will substitute %PUBLIC_URL% with a correct absolute path so your project works even if you use client-side routing or host it at a non-root URL.
In JavaScript code, you can use process.env.PUBLIC_URL for similar purposes:
render() {
// Note: this is an escape hatch and should be used sparingly!
// Normally we recommend using `import` for getting asset URLs
// as described in “Adding Images and Fonts” above this section.
return <img src={process.env.PUBLIC_URL + '/img/logo.png'} />;
}Keep in mind the downsides of this approach:
- None of the files in
publicfolder get post-processed or minified. - Missing files will not be called at compilation time, and will cause 404 errors for your users.
- Result filenames won’t include content hashes so you’ll need to add query arguments or rename them every time they change.
However, it can be handy for referencing assets like manifest.webmanifest from HTML, or including small scripts like pace.js outside of the bundled code.
Note that if you add a <script> that declares global variables, you also need to read the next section on using them.
Note: this feature is available with
[email protected]and higher.
Your project can consume variables declared in your environment as if they were declared locally in your JS files. By
default you will have NODE_ENV defined for you, and any other environment variables starting with
REACT_APP_. These environment variables will be defined for you on process.env. For example, having an environment
variable named REACT_APP_SECRET_CODE will be exposed in your JS as process.env.REACT_APP_SECRET_CODE, in addition
to process.env.NODE_ENV.
Note: Changing any environment variables will require you to restart the development server if it is running.
These environment variables can be useful for displaying information conditionally based on where the project is deployed or consuming sensitive data that lives outside of version control.
First, you need to have environment variables defined. For example, let’s say you wanted to consume a secret defined
in the environment inside a <form>:
render() {
return (
<div>
<small>You are running this application in <b>{process.env.NODE_ENV}</b> mode.</small>
<form>
<input type="hidden" defaultValue={process.env.REACT_APP_SECRET_CODE} />
</form>
</div>
);
}During the build, process.env.REACT_APP_SECRET_CODE will be replaced with the current value of the REACT_APP_SECRET_CODE environment variable. Remember that the NODE_ENV variable will be set for you automatically.
When you load the app in the browser and inspect the <input>, you will see its value set to abcdef, and the bold text will show the environment provided when using npm start:
<div>
<small>You are running this application in <b>development</b> mode.</small>
<form>
<input type="hidden" value="abcdef" />
</form>
</div>Having access to the NODE_ENV is also useful for performing actions conditionally:
if (process.env.NODE_ENV !== 'production') {
analytics.disable();
}The above form is looking for a variable called REACT_APP_SECRET_CODE from the environment. In order to consume this
value, we need to have it defined in the environment. This can be done using two ways: either in your shell or in
a .env file.
Defining environment variables can vary between OSes. It's also important to know that this manner is temporary for the life of the shell session.
set REACT_APP_SECRET_CODE=abcdef&&npm start(Note: the lack of whitespace is intentional.)
REACT_APP_SECRET_CODE=abcdef npm startNote: this feature is available with
[email protected]and higher.
To define permanent environment variables, create a file called .env in the root of your project:
REACT_APP_SECRET_CODE=abcdef
These variables will act as the defaults if the machine does not explicitly set them.
Please refer to the dotenv documentation for more details.
Note: If you are defining environment variables for development, your CI and/or hosting platform will most likely need these defined as well. Consult their documentation how to do this. For example, see the documentation for Travis CI or Heroku.
Check out this tutorial for instructions on integrating an app with a Node backend running on another port, and using fetch() to access it. You can find the companion GitHub repository here.
Note: this feature is available with
[email protected]and higher.
People often serve the front-end React app from the same host and port as their backend implementation.
For example, a production setup might look like this after the app is deployed:
/ - static server returns index.html with React app
/todos - static server returns index.html with React app
/api/todos - server handles any /api/* requests using the backend implementation
Such setup is not required. However, if you do have a setup like this, it is convenient to write requests like fetch('/api/todos') without worrying about redirecting them to another host or port during development.
To tell the development server to proxy any unknown requests to your API server in development, add a proxy field to your package.json, for example:
"proxy": "http://localhost:4000",This way, when you fetch('/api/todos') in development, the development server will recognize that it’s not a static asset, and will proxy your request to http://localhost:4000/api/todos as a fallback. The development server will only attempt to send requests without a text/html accept header to the proxy.
Conveniently, this avoids CORS issues and error messages like this in development:
Fetch API cannot load http://localhost:4000/api/todos. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:3000' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Keep in mind that proxy only has effect in development (with npm start), and it is up to you to ensure that URLs like /api/todos point to the right thing in production. You don’t have to use the /api prefix. Any unrecognized request without a text/html accept header will be redirected to the specified proxy.
Currently the proxy option only handles HTTP requests, and it won’t proxy WebSocket connections.
If the proxy option is not flexible enough for you, alternatively you can:
- Enable CORS on your server (here’s how to do it for Express).
- Use environment variables to inject the right server host and port into your app.
HTTPS=true npm startNote that the server will use a self-signed certificate, so your web browser will almost definitely display a warning upon accessing the page.
Note: this feature is available with
[email protected]and higher.
Read the migration guide to learn how to enable it in older projects!
Create React App uses Jest as its test runner. To prepare for this integration, we did a major revamp of Jest so if you heard bad things about it years ago, give it another try.
Jest is a Node-based runner. This means that the tests always run in a Node environment and not in a real browser. This lets us enable fast iteration speed and prevent flakiness.
While Jest provides browser globals such as window thanks to jsdom, they are only approximations of the real browser behavior. Jest is intended to be used for unit tests of your logic and your components rather than the DOM quirks.
We recommend that you use a separate tool for browser end-to-end tests if you need them. They are beyond the scope of Create React App.
Jest will look for test files with any of the following popular naming conventions:
- Files with
.jssuffix in__tests__folders. - Files with
.test.jssuffix. - Files with
.spec.jssuffix.
The .test.js / .spec.js files (or the __tests__ folders) can be located at any depth under the src top level folder.
We recommend to put the test files (or __tests__ folders) next to the code they are testing so that relative imports appear shorter. For example, if App.test.js and App.js are in the same folder, the test just needs to import App from './App' instead of a long relative path. Colocation also helps find tests more quickly in larger projects.
When you run npm test, Jest will launch in the watch mode. Every time you save a file, it will re-run the tests, just like npm start recompiles the code.
The watcher includes an interactive command-line interface with the ability to run all tests, or focus on a search pattern. It is designed this way so that you can keep it open and enjoy fast re-runs. You can learn the commands from the “Watch Usage” note that the watcher prints after every run:
By default, when you run npm test, Jest will only run the tests related to files changed since the last commit. This is an optimization designed to make your tests runs fast regardless of how many tests you have. However it assumes that you don’t often commit the code that doesn’t pass the tests.
Jest will always explicitly mention that it only ran tests related to the files changed since the last commit. You can also press a in the watch mode to force Jest to run all tests.
Jest will always run all tests on a continuous integration server or if the project is not inside a Git or Mercurial repository.
To create tests, add it() (or test()) blocks with the name of the test and its code. You may optionally wrap them in describe() blocks for logical grouping but this is neither required nor recommended.
Jest provides a built-in expect() global function for making assertions. A basic test could look like this:
import sum from './sum';
it('sums numbers', () => {
expect(sum(1, 2)).toEqual(3);
expect(sum(2, 2)).toEqual(4);
});All expect() matchers supported by Jest are extensively documented here.
You can also use jest.fn() and expect(fn).toBeCalled() to create “spies” or mock functions.
There is a broad spectrum of component testing techniques. They range from a “smoke test” verifying that a component renders without throwing, to shallow rendering and testing some of the output, to full rendering and testing component lifecycle and state changes.
Different projects choose different testing tradeoffs based on how often components change, and how much logic they contain. If you haven’t decided on a testing strategy yet, we recommend that you start with creating simple smoke tests for your components:
import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';
it('renders without crashing', () => {
const div = document.createElement('div');
ReactDOM.render(<App />, div);
});This test mounts a component and makes sure that it didn’t throw during rendering. Tests like this provide a lot value with very little effort so they are great as a starting point, and this is the test you will find in src/App.test.js.
When you encounter bugs caused by changing components, you will gain a deeper insight into which parts of them are worth testing in your application. This might be a good time to introduce more specific tests asserting specific expected output or behavior.
If you’d like to test components in isolation from the child components they render, we recommend using shallow() rendering API from Enzyme. You can write a smoke test with it too:
npm install --save-dev enzyme react-addons-test-utilsimport React from 'react';
import { shallow } from 'enzyme';
import App from './App';
it('renders without crashing', () => {
shallow(<App />);
});Unlike the previous smoke test using ReactDOM.render(), this test only renders <App> and doesn’t go deeper. For example, even if <App> itself renders a <Button> that throws, this test will pass. Shallow rendering is great for isolated unit tests, but you may still want to create some full rendering tests to ensure the components integrate correctly. Enzyme supports full rendering with mount(), and you can also use it for testing state changes and component lifecycle.
You can read the Enzyme documentation for more testing techniques. Enzyme documentation uses Chai and Sinon for assertions but you don’t have to use them because Jest provides built-in expect() and jest.fn() for spies.
Here is an example from Enzyme documentation that asserts specific output, rewritten to use Jest matchers:
import React from 'react';
import { shallow } from 'enzyme';
import App from './App';
it('renders welcome message', () => {
const wrapper = shallow(<App />);
const welcome = <h2>Welcome to React</h2>;
// expect(wrapper.contains(welcome)).to.equal(true);
expect(wrapper.contains(welcome)).toEqual(true);
});All Jest matchers are extensively documented here.
Nevertheless you can use a third-party assertion library like Chai if you want to, as described below.
We recommend that you use expect() for assertions and jest.fn() for spies. If you are having issues with them please file those against Jest, and we’ll fix them. We intend to keep making them better for React, supporting, for example, pretty-printing React elements as JSX.
However, if you are used to other libraries, such as Chai and Sinon, or if you have existing code using them that you’d like to port over, you can import them normally like this:
import sinon from 'sinon';
import { expect } from 'chai';and then use them in your tests like you normally do.
Note: this feature is available with
[email protected]and higher.
If your app uses a browser API that you need to mock in your tests or if you just need a global setup before running your tests, add a src/setupTests.js to your project. It will be automatically executed before running your tests.
For example:
const localStorageMock = {
getItem: jest.fn(),
setItem: jest.fn(),
clear: jest.fn()
};
global.localStorage = localStorageMockYou can replace it() with xit() to temporarily exclude a test from being executed.
Similarly, fit() lets you focus on a specific test without running any other tests.
Jest has an integrated coverage reporter that works well with ES6 and requires no configuration.
Run npm test -- --coverage (note extra -- in the middle) to include a coverage report like this:
Note that tests run much slower with coverage so it is recommended to run it separately from your normal workflow.
By default npm test runs the watcher with interactive CLI. However, you can force it to run tests once and finish the process by setting an environment variable called CI.
When creating a build of your application with npm run build linter warnings are not checked by default. Like npm test, you can force the build to perform a linter warning check by setting the environment variable CI. If any warnings are encountered then the build fails.
Popular CI servers already set the environment variable CI by default but you can do this yourself too:
- Following the Travis Getting started guide for syncing your GitHub repository with Travis. You may need to initialize some settings manually in your profile page.
- Add a
.travis.ymlfile to your git repository.
language: node_js
node_js:
- 4
- 6
cache:
directories:
- node_modules
script:
- npm test
- npm run build
- Trigger your first build with a git push.
- Customize your Travis CI Build if needed.
set CI=true&&npm testset CI=true&&npm run build(Note: the lack of whitespace is intentional.)
CI=true npm testCI=true npm run buildThe test command will force Jest to run tests once instead of launching the watcher.
If you find yourself doing this often in development, please file an issue to tell us about your use case because we want to make watcher the best experience and are open to changing how it works to accommodate more workflows.
The build command will check for linter warnings and fail if any are found.
By default, the package.json of the generated project looks like this:
// ...
"scripts": {
// ...
"test": "react-scripts test --env=jsdom"
}If you know that none of your tests depend on jsdom, you can safely remove --env=jsdom, and your tests will run faster.
To help you make up your mind, here is a list of APIs that need jsdom:
- Any browser globals like
windowanddocument ReactDOM.render()TestUtils.renderIntoDocument()(a shortcut for the above)mount()in Enzyme
In contrast, jsdom is not needed for the following APIs:
TestUtils.createRenderer()(shallow rendering)shallow()in Enzyme
Finally, jsdom is also not needed for snapshot testing. Longer term, this is the direction we are interested in exploring, but snapshot testing is not fully baked yet so we don’t officially encourage its usage yet.
Snapshot testing is a new feature of Jest that automatically generates text snapshots of your components and saves them on the disk so if the UI output changes, you get notified without manually writing any assertions on the component output.
This feature is experimental and still has major usage issues so we only encourage you to use it if you like experimental technology. We intend to gradually improve it over time and eventually offer it as the default solution for testing React components, but this will take time. Read more about snapshot testing.
If you use Visual Studio Code, there is a Jest extension which works with Create React App out of the box. This provides a lot of IDE-like features while using a text editor: showing the status of a test run with potential fail messages inline, starting and stopping the watcher automatically, and offering one-click snapshot updates.
npm run build creates a build directory with a production build of your app. Set up your favourite HTTP server so that a visitor to your site is served index.html, and requests to static paths like /static/js/main.<hash>.js are served with the contents of the /static/js/main.<hash>.js file. For example, Python contains a built-in HTTP server that can serve static files:
cd build
python -m SimpleHTTPServer 9000If you're using Node and Express as a server, it might look like this:
const express = require('express');
const path = require('path');
const app = express();
app.use(express.static('./build'));
app.get('/', function (req, res) {
res.sendFile(path.join(__dirname, './build', 'index.html'));
});
app.listen(9000);Create React App is not opinionated about your choice of web server. Any static file server will do. The build folder with static assets is the only output produced by Create React App.
However this is not quite enough if you use client-side routing. Read the next section if you want to support URLs like /todos/42 in your single-page app.
If you use routers that use the HTML5 pushState history API under the hood (for example, React Router with browserHistory), many static file servers will fail. For example, if you used React Router with a route for /todos/42, the development server will respond to localhost:3000/todos/42 properly, but an Express serving a production build as above will not.
This is because when there is a fresh page load for a /todos/42, the server looks for the file build/todos/42 and does not find it. The server needs to be configured to respond to a request to /todos/42 by serving index.html. For example, we can amend our Express example above to serve index.html for any unknown paths:
app.use(express.static('./build'));
-app.get('/', function (req, res) {
+app.get('/*', function (req, res) {
res.sendFile(path.join(__dirname, './build', 'index.html'));
});Now requests to /todos/42 will be handled correctly both in development and in production.
If you run npm test and the console gets stuck after printing react-scripts test --env=jsdom to the console there might be a problem with your Watchman installation as described in facebookincubator/create-react-app#713.
We recommend deleting node_modules in your project and running npm install (or yarn if you use it) first. If it doesn't help, you can try one of the numerous workarounds mentioned in these issues:
It is reported that installing Watchman 4.7.0 or newer fixes the issue. If you use Homebrew, you can run these commands to update it:
watchman shutdown-server
brew update
brew reinstall watchman
You can find other installation methods on the Watchman documentation page.
If this still doesn't help, try running launchctl unload -F ~/Library/LaunchAgents/com.github.facebook.watchman.plist.
There are also reports that uninstalling Watchman fixes the issue. So if nothing else helps, remove it from your system and try again.
It is reported that npm run build can fail on machines with no swap space, which is common in cloud environments. If the symptoms are matching, consider adding some swap space to the machine you’re building on, or build the project locally.