Break down the work into smaller modules and develop pipelines (consists of module) for the work. Be sure to save (important) intermediate results so that you can resume failed modules withouting reruning succeeded ones.

Manage your project in GitHub and use issues to manage tasks to do and their priorities …