Separate ‘sources’ repository?

[fruchti]

There should be a place for 3D design files for the STEP files included in the pool, similar to KiCad's packages3D-source. Personally, I'd also move the part-generating scripts currently residing in this repository there.

Everything could be kept in one repository which would make keeping everything in sync easier. On the other hand, since most users won't care about all pool source files, it would just mean a lot of wasted space for most installations.

[carrotIndustries]

Oh well, the sources for the generated parts are indeed larger than expected. For now, I think that keeping everything in one repo outweighs the benefit of a smaller repo.

Another approach might be to gzip the source files extracted from the manufacturer's websites as they're often too big so sensibly diff in the first place. Gzipping the erj.json source file reduced its size from 28M to 450k.

[fruchti]

I've never tried it but it's probably possible to move all large files and their histories to a separate repository? That way we can delay this decision without risking a permanently huge repository.

Where should the 3D design files (FreeCAD models, OpenSCAD scripts, etc.) go?

[fbruetting]

Not sure if I understand this correctly, but there’s git-lfs for excluding (especially large) files from a Git repository. When you clone such a repo, you then don’t get the complete history of those excluded files, but just the checkout of the most recent version.

[atoav]

These are the sizes for my own pool and as we can see the majority of the size is in 3d_models, scripts and pool.db. The scripts are nice to have documented, but really don't need to be redistributed with the pool.

The 3D-parts are a different thing, how we should store them depends on how we wanna deal with them in the software. They are naturally big, but as you might only use a fraction of the parts anyways (and not everyone will open the 3D view at all times) maybe it would be smart to handle these in a different way to the way we do now.

Imagine 3 different choices when initially loading the pool:

• Download all the 3D files (if you don't mind downloading and storing the >100MB)
• Lazy load only the ones needed in the moment the 3D view opens
• Don't load them at all

Now that I think of it: downloading the pool (sans 3D parts) first and then downloading the 3D files in a background-thread without blocking the UI would probably be even better, because the 3D-files are a nice-to-have and not something that breaks the core functionality. In any way this seems to suggest that the proposal @fruchti made to offload these files to a different repo makes sense.