​

Development Guide

This guide will help you set up OpenElectricity for local development. The project consists of two main components:

• A FastAPI web API server
• An Arq background worker for processing tasks

​

Prerequisites

​

Installing uv

We use uv as our package manager and virtual environment tool. It’s significantly faster than pip and provides better dependency resolution.

Install uv using one of these methods:

macOS/Linux:

curl -LsSf https://astral.sh/uv/install.sh | sh

​

Project Setup

1. Clone the repository:

git clone https://github.com/opennem/opennem.git
cd opennem

2. Create the virtual environment and install dependencies:

uv install
source .venv/bin/activate  # On Windows use: .venv\Scripts\activate

​

Core Dependencies

OpenElectricity uses several key libraries:

• FastAPI - Modern web framework for building APIs
• SQLAlchemy 2.0 - SQL toolkit and ORM
• asyncpg - Async PostgreSQL driver
• Alembic - Database migration tool
• Arq - Async job queue and worker
• Pydantic - Data validation using Python type annotations
• uvicorn - ASGI server for running the API
• TimescaleDB - Time-series database extension for PostgreSQL
• Redis - In-memory data store used by Arq

​

Running the Application

The application consists of two main processes that need to be run:

​

1. API Server

The FastAPI application serves the REST API endpoints. Run it with:

uv run api

This will start the API server at http://localhost:8000

The OpenAPI documentation will be available at:

• http://localhost:8000/docs (Swagger UI)

​

2. Background Worker

The Arq worker processes background tasks like data ingestion and exports. Run it with:

uv run worker

​

Development Database

1. Install PostgreSQL 17 locally

2. Create a database:

createdb opennem

3. Run migrations:

uv run alembic upgrade head

​

Local Services

We provide a Docker Compose configuration for running required services locally. Start the services with:

docker-compose up -d

This will start:

• PostgreSQL 17 with TimescaleDB extension
• Redis server for the task queue

​

Environment Setup

1. Copy the example environment file:

cp .env.example .env

2. Edit the .env file with your local settings. The example file contains all required variables with sensible defaults for local development.

​

Configuration Settings

All application settings are defined in opennem/settings_schema.py and are loaded from environment variables. Key settings include:

• Database connection details
• Redis configuration
• API settings
• Authentication settings
• Logging configuration

You can view all available settings and their documentation in the schema file. Each setting can be overridden using environment variables with the OPENNEM_ prefix.

Example settings from settings_schema.py:

​

Code Quality Tools

We use several tools to maintain code quality:

• Ruff - Fast Python linter and formatter
• mypy - Static type checker
• pytest - Testing framework

Run the quality checks:

​

API Documentation

The API documentation is automatically generated from the OpenAPI schema. You can view it at:

• Local development: http://localhost:8000/docs
• Production: https://api.openelectricity.org.au/docs

​

Getting Help

If you need help:

1. Check the project documentation
2. Open an issue on GitHub
3. Join our community discussions

​

Contributing

1. Create a new branch for your feature
2. Make your changes
3. Run the test suite
4. Submit a pull request

Please follow our coding standards:

• Use type hints for all function parameters and returns
• Write docstrings for all functions and modules
• Follow PEP 8 style guidelines
• Write tests for new functionality