[GSOC 2020] My Proposal for a parallel test runner

Hello there,

I would much appreciate it if you could go through my GSOC proposal and give suggestions. Honestly, after going through proposals from previous years, I’m not sure if the amount of detail under Schedule is enough.

Thanks
Abhijeet

1 Like

Hi Abhijeet,

is there any impact of this proposal to running test in parallel on Oracle? Currently Django does not support it (I failed to find why). Will parallel testing on Oracle be supported when your proposal is implemented?

Thank you.

Hi,

This proposal will not impact running tests in parallel on Oracle. I believe the reason why there is no support for it right now is because it isn’t possible to clone databases to run tests in parallel. From what little research I did, it isn’t possible without external utilities (unlike utilities like mysqldump or template databases in case of MySQL and PostgreSQL).

The alternative would be to simply create multiple databases. I’m assuming this would result in a huge performance hit.

I’d be more than glad to amend my proposal and add support for Oracle DB.

Thanks,
Abhijeet.

2 Likes

It seems I was completely wrong.
Tim Graham pointed me to this comment on the original Github PR.

Hi,

I spent the better part of last week remaking my entire proposal and expanding the original project to include adding support of Oracle as well.

Here is the link.

I’d highly appreciate it if y’all could go through the proposal and leave feedback.

@carltongibson @aaugustin

Thanks. :slight_smile:

Hi @abhijeetviswa.

The proposal looks OK. The PR is a nice touch. :slightly_smiling_face:

If I were to make one comment, I’d say to flesh out the Overview section of the part where you discuss the spawn method.

If you can explain at a high level what the tasks we need to complete in the init worker phase are, then that shows you’ve really grasped what’s needed to be done.

That’s just a thought.

Not sure the “advantages” and “disadvantages” sections add too much: we want to do this! :slightly_smiling_face: The advantage is faster tests.

Hopefully that helps. Good luck with the submission.

Carlton

Hey ,

Thanks for the reply.

I’ll flesh out the overview section. I wasn’t sure how to best explain it. That was why the PR/POC.

As for the advantages and disadvantages, I added that in because it was their in last year’s proposal as well. :sweat_smile: If it isn’t a problem, I’ll leave it in there.

Thank you.

EDIT: I’ve edited the original gist, @carltongibson . Please go through it when you have the time. Thanks.

1 Like

Hey @abhijeetviswa, this is a very well written and detailed proposal. Good luck with the submission!

I don’t have much to add, other than to say that Oracle can be fickle, and so do you have a plan if one of the Oracle-specific steps turns out to be much harder or more complex than anticipated? If not, do you have an idea of which of the 4 Oracle weeks might be the most challenging?

Hi @orf,

Thanks.

I understood the fickle nature when I was working on writing this proposal. It took me 3 days just to get it running on my machine. :joy:

I had done some testing with both the expdp/impdp utilities as well as with PL/SQL API (using the SQLPlus CLI) and both of them worked on Oracle 12.2 (the last supported version on Django 3.0). Since I haven’t tested it using Python or Django, that’s where I expect I’ll be spending the most amount of time, especially making dumps work while connected to a remote database. This is also the reason why I have scheduled 2 weeks just to implement importing and exporting the dumps between the different test user schemas.

Thanks.

Hey y’all

I realized today morning that due to the Covid-19 pandemic the GSoC timeline has. I had to change a couple of dates around and also reworded some parts of the proposal a bit better.

The updated version of the proposal can be found here (It is still the same link) .

This is the final draft of the proposal which I will be uploading to the GSOC portal as well. I’d like to thank everyone here who helped me with writing the proposal and I hope that I’ll be able to work with you on this project.

Thanks and stay safe.