File upload on mobile connection

Hey,

I am trying to solve problems with file upload on mobile connections. How would you solve the problem of bad connections and resuming uploads. Do I have to create a mobile app or something or can I create something with JS and service workers? Can you give me some resources on this topic?

Regards,

Max

Hi Max,

Assuming you’ve already compressed the files and have worked your requirements to reduce the size and number of files, I’d start looking into a way to chunk the files in to pieces, then focus on getting each of those chunks to the server. Here’s a SO question that has a few solutions on how to do it. I’ve linked to the one that specifically breaks it into a single binary file that gets uploaded.

If that still has problems, I would consider creating a persistent record for the file upload. It would have to be keyed off of something that could uniquely identify the file. Perhaps a combination of the name, size and a quick hash like md5? I’m not sure if that’s all possible in JS, I haven’t dealt with file manipulation much there.

Once you have the key for it persisted on the server side, you would store each segment of the data uploaded and accumulate it until you have the entire file. If you’re using a hash of the file, you can rerun that to attempt to verify the file isn’t corrupted from the upload process.

To me, this is a very heavy handed approach to solving your problem, but if the connections are extremely spotty, the file size is large or there’s a large number of uploads that are done in this environment, it may be warranted.

One way the process could look like is the following:

  1. Client sends a message to the server indicating it wants to upload a file, includes the size, hash, other meta information.
  2. Server persists the meta information, including a cursor to track where the upload is at, returns cursor start point and hash.
  3. Client attempts to send a chunk for the given hash, includes current cursor value.
  4. Server records chunk for the given cursor, responds with a message with current cursor and success indicator
  5. If client gets a success, it moves onto the next chunk with the new cursor. If the client errors, it reattempts the error case according to some retry/backoff strategy
  6. Rinse and repeat until file is uploaded.
  7. Client indicates to the server that file is uploaded.
  8. Server reassembles the file and validates according to the hash.