I created a tool for copying large and complex model hierarchies

Show & Tell
1

Hey folks!

As per title, I created new lib and wanted to share it with you

Github - GitHub - abondar/django-copyist: Tool for precise and efficient django model copying

Docs - django-copyist — django-copyist 0.1.0 documentation

TLDR of key features:

  • Copying process is described through declarative config, so you see all process description at a glance, and you don’t need to modify your models

  • Handles all relations between models, including M2M, so you are able to copy whole hierarchy in one transaction

  • Has good performance, as it doesn’t create models by recursively going through nested instances one by one (tested on a project with dozens of thousands records copied per request)

  • Extremely flexible, allowing different kind of tinkering with data before, during and after main copy process

You can read more at the docs or on github, so please check it out

1 Like
2

how does this differ from dumpdata/loaddata?

when should I use it?

3

dumpdata/loaddata copies data just as serialized rows, which usually focused on copying data between databases/creating backups.

If we look at case where we want to copy several interrelated models within one database - these commands will not work properly for you, as they do not offer API for precise filtering of dataset, so you end up with whole table serialised

But copyist offers such tools for filtering and ignoring certain rows, so you have much more precise control of what’s going on. More advantages are;

  • Per field control copy - you can describe which fields should be moved to new models, and which should use default values
  • You can, not only take values for fields from origin model instance, but also take it from input or match ids with previously copied related data
  • Running custom logic before/after copy to prepare/postprocess your data
  • Clean python API, allowing you to setup this copy process as part of your service functionality, customised by user input

You can ready more in usecase oriented part of documentation, where (almost) each section describes certain usecase and tool for it
https://abondar.github.io/django-copyist/overview.html

4

That sounds convincing, I’ll keep it in mind for whenever I have to do something similar!

good job! awesome docs btw! :tada: