swdata package

Submodules

swdata.adapters module

swdata.adapters.request_sw_data(url)

Adapter function for getting data from SWAPI url

swdata.adapters.results_generator(url: str) → Generator[Dict, None, None]

Result generator that will automatically request next result page until there are no more results

swdata.people module

These are people provider and transformation classes.

class swdata.people.PeopleProvider(url: str = 'https://swapi.dev/api/people/')

Bases: object

Simple api adapter for fetching people data from SWAPI.

get_people() → Generator[Dict, None, None]

url: str = 'https://swapi.dev/api/people/'

class swdata.people.PeopleSaver(transformer: swdata.people.PeopleTransformator = <factory>, provider: swdata.people.PeopleProvider = <factory>)

Bases: object

An aggregate for handling src people data and saving reports.

provider: swdata.people.PeopleProvider

save(directory: str) → swdata.reports.Report

transformer: swdata.people.PeopleTransformator

class swdata.people.PeopleTransformator(planets: swdata.planets.PlanetProvider = <factory>)

Bases: object

Transformation pipeline component to enrich People data and replace home world reference with a name.

fields = ('name', 'height', 'mass', 'hair_color', 'skin_color', 'eye_color', 'birth_year', 'gender', 'homeworld', 'edited', 'url')

planets: swdata.planets.PlanetProvider

transform_people(people: Iterator[Dict]) → Iterator[Tuple]

swdata.planets module

These are planet data providers.

ATM there is only 60 planets in the pool so this is not a problem to keep them in memory. When circumstances change, and the pool of planets would increase significantly, we can implement a more efficient data provider that would build index an index only once store it only update missing planets when they are requested.

class swdata.planets.EagerPlanetProvider

Bases: swdata.planets.PlanetProvider

Eager provider loading all planets into memory upfront.

get_name(url)

planets

class swdata.planets.LazyPlanetProvider

Bases: swdata.planets.PlanetProvider

Lazy and slow planet provider usable only when the number queries planets is small in comparison to the total number of available planets.

CACHED_PLANETS = 100

CACHE_TIME = 3600

get_name(url)

class swdata.planets.PlanetProvider

Bases: abc.ABC

abstract get_name(url: str)

swdata.reports module

class swdata.reports.Report(path: str, when_saved: datetime.datetime)

Bases: swdata.reports.ReportViewer

Report metadata DTO

when_saved: datetime.datetime

class swdata.reports.ReportViewer(path: str)

Bases: object

Data viewer for reports saved on disc

distinct(*columns)

path: str

swdata.reports.save_csv(path: str, data: Iterator)

Utility function to save data to a CSV file.

swdata.reports.save_report(directory: str, data: Iterator, timestamp_fmt='%Y-%m-%d-%H%M%S') → swdata.reports.Report

Utility function create a report file with timestamp in the name

swdata.settings module

Module contents

This a public API for the swdata package.

Everything beside classes declared in this module should be considered private and subject to change without notice. This is an architectural decision to give this module an autonomy over it’s design and implementation.

ATM we are using poor man’s dependency injection delivered by dataclass fields. Preferably we should use a proper Inversion Of Control container engine that would give use the flexibility of defining a default types and setup mappings like the one between PlanetProvider abstract class and a concrete implementation of EagerPlanetProvider — mappings like this should come from the container setup not class declarations.

ATM the control over injected instances is delegated to the field declarations and setup by default_factory.

class swdata.SWPeople(_saver: swdata.people.PeopleSaver = <factory>)

Bases: object

Gateway class for reporting functionality over SW People data.

create_collection(directory='reports') → swdata.reports.Report

static get_distinct(path, *columns)

static get_people_data(path, start=0, stop=10)