Connecting Your Data

To use Lookout, you must write a Connector script that retrieves your health data and converts it to the records.json format used by Lookout. Lookout always looks for records.json files in the site/data directory.

The records.json format

Lookout reads health data as key-value pairs stored in records.json, which is simply an array of JSON objects where each object represents a health record. For example:

[
  {
    "date": "2026-01-01",
    "disease-a": "positive",
    "district": "ES",
    "age": 42,
  },
  {
    "date": "2026-01-03",
    "disease-a": "negative",
    "disease-b": "inconclusive",
    "district": "ES",
    "count": 4,
    "age": 23,
  },
  // ...
]

Each JSON object in the array must contain a date and at least one pathogen result. In the example above:

  • The first health record’s date is 2026-01-01 and it contains a pathogen result for disease-a that is positive.
  • The second health record’s date is 2026-01-03, and it accounts for four cases that contains two pathogen results: disease-a was negative and disease-b was inconclusive.

records.json Field Description Value
date required The date attributed to the record.

Note: Lookout is best suited for datasets with complete date information and expects fully specified dates. For datasets with alternate date types, you must format your dates correctly in your connector script (YYYY-MM-DD) for them to be recognized by Lookout.		 YYYY-MM-DD
disease-a or disease-b, etc. required Test result for pathogen disease-a or disease-b, etc.

Note: The specified pathogen field must match one of the diseases specified in the "pathogens" field in config.json.		 Any string (e.g. "positive", "negative", or "suspected")
count optional Number of cases the record accounts for. Defaults to 1. integer
Custom fields optional Additional information about the record, such as patient ID, demographic information, geographic data, symptoms, etc. string, boolean, integer,float, or array

Not every record is required to have the same fields. If some records contain information that other records are missing, the missing information can be omitted or entered as null.

For advanced features, such as multiple datasets, sequencing, or multiple tests for a pathogen, see the full records.json reference.

Writing a Connector

A Connector is the script that converts your data into a records.json file. You maintain it yourself, so you can add new data fields, change formats, or make any other edits.

We have provided some barebones Connectors in Python for different types of source data files, which you can customize:

  • .csv and .tsv files to records.json.
    • This connector is included in the Lookout template at connectors/csv-connector-template.py.
  • SORMAS data to records.json
  • (coming soon) DHIS2 data to records.json

A Connector should output a records.json file in site/data, as that is where Lookout will look for data files. Other than that, you can infinitely customize a Connector, as it is just a script, and you can also write your Connectors in any programming language you wish.

Running a Connector

By convention, we have placed Connectors in a connectors folder on the same level as the site folder, and our demos look for source data files (like .csvs) in the connectors folder. However, you can configure your Connector to pull from wherever your data is stored.

Our demos are all Python scripts. Run them with:

python3 <script-name>.py

Continually Updating Data

A Lookout site can be updated simply by updating records.json with new data. Depending on your data pipelines, these updates can either be done manually (copying in new data and running the script) or automatically (e.g. setting up a cron job to run the script periodically).

Before moving on, check your records.json file to make sure your records include the required date and pathogen fields.

Once your records are all set, it’s time to Display Your Data.