I am interested in creating a code practice site where I can execute code from various languages (think leetcode.com or hackerrank.com).
Initially, I’d like to work with Python and have code vetted using PyTest.
If I can get Python working, I’d like to be able to execute Golang and Java as well.
The finished product here would be a Django site where I can pose a challenge like “Create an array containing even numbers from 1-100.”
The user would write Python syntax as a function, I’d source the function, execute a PyTest against the function, and see if the desired object gets returned. If the function performs as desired, the site notifies the user, otherwise it advises that the function written did not pass the test.
My questions are:
- Is something like this possible with Django?
- If so, what does the high-level layout/stack look like?
Thanks in advance for any help!
I know my knowledge of this is greatly out-of-date, but the last time I looked at this it was really difficult to create a secure sandbox in which the code is being run that isn’t able to be hacked / reveal unintended information.
The python wiki has a couple of pages identifying some of the issues involved - Sandboxed Python and How can I run an untrusted Python script safely (i.e. Sandbox) should give you some information about why this is a difficult topic.
Also, there’s this blog post by Ned Batchelder - Eval is really dangerous that is a really good read. (Actually, his entire blog is a really good read.)
All that aside for the moment, I could see (possibly) being able to set up a set of docker containers that have very strict limits around them into which you dispatch the task. You could even go so far as set up lightweight virtual machines that would be even more restricted.
The key idea here is that for every method that someone has identified to limit functionality, someone else has generally found a way around it, so any mechanism you want to employ to provide this facility has to be as bullet-proof as you can possibly make it.
Java would be easier because it’s already designed to run in a limited resource sandbox (the JVM) and there are security mechanisms you can apply beyond that to limit the adverse effects of anything being run.
@KenWhitesell - Thank you so much for your reply.
I knew security would be a challenging issue when conceptualizing this problem. These articles have been a great read.
I am not a developer by trade, so forgive me if the answer to this question is obvious.
I wonder, from Python’s perspective, how would untrusted code be received for testing?
Would you write a service that can receive the untrusted code as a string?
Or perhaps have the service remotely read the untrusted code from a file that is server-side to Django?
If I were implementing something like this, I would have my sandbox running as a service listener that would accept the code as a string - I wouldn’t want to be dealing with a lot of file management from generic web form submissions. (I would also ensure that I had strict size limits on the submitted form.)
I’d also ensure that the listener was restarted after every run to prevent someone from polluting the runtime environment and affecting the next person using that same listener. So I might use something as simple as the built-in http.server to listen for that single request, execute it, then exit. I’d wrapper that inside whatever sandbox management tool you’re using that would recognize that the process had ended and then restart it. If this process were a docker container, I believe it would be able to “automatically” restart.
This is all just off-the-cuff ideas - as they say, the devil’s in the details…