Custom data on individual BatchItems

[mwdiaz]

Uploady accepts custom arbitrary data as a userData prop on the main Uploader component, but we also have a couple different scenarios where it would be nice to be able to store data on individual BatchItems.

Our primary use case is being able to create more globally unique identifiers that will both be included as a header on actual upload requests, as well as any log messages we generate on the client specific to a BatchItem. This allows us to easily correlate server-side logs across disparate systems with logs sent from the browser. The id that Uploady creates for BatchItems is unique within a given pageload (since it merely increments a counter stored local to a module), so that isn't something we can use. By way of example, here is how we currently generate a "unique enough" id in our legacy uploader: Math.random().toString(36).substring(2). This gives us something that is more or less guaranteed to be globally unique across all our users within the relatively short windows of time (seconds to minutes, generally) while a file is being uploaded.

For the aforementioned id case, a possible alternative to allowing user data on BatchItems would be to accept a custom id generator function (or similar), but this isn't our only use case for storing custom data on BatchItems. Another example might be tracking a failure reason/code on individual BatchItems for later displaying a message in the UI. Currently, we accomplish this sort of thing by maintaining a separate object in local state that maps a BatchItem id to message.

I'm not entirely sure what form this might take, since in many cases where we'd want to add/update this custom data (for example in a useItem*Listener hook), it isn't clear to me how to facilitate returning a potentially modified BatchItem while also maintaining the existing semantics of the hook. I'm curious to hear any thoughts/recommendations about these use cases and whether it would be a good idea to support them natively; or if it's better left as something for consumers to implement as required.

Thanks for all your hard work on this awesome library!

[yoavniran]

Hi @mwdiaz
thanks for the kind words and detailed request.

I have been, and still am, reluctant to make Uploady a state storage for its hosting applications. I can say that I'm still conflicted even over the userData prop.

On the one hand, it will be fairly trivial (at this point) to add a calculator function that will be called upon batch creation (similar to the fileFilter method) for each item and assign to a userData prop on the batchItem object whatever returns.

On the other hand, I'm not too happy to make Uploady carry around unrelated data and perhaps create complexities in the future and/or block desired changes later on.

So before I rush to implement, I'd like to understand better whether a simple, user-land scenario can't achieve the desired outcome - What about using useItemStartListener to store a map (in your app's state, which i assume already is in place) between the Uploady item id and custom data you wish to hold for it. Something like:

  useItemStartListenter(({ id }) => {
         setState((prev) => { ...prev, [id]: { identifier: Math.random().toString(36).substring(2), ... } })
   });

use as header:

 useRequestPreSend(({ items, options }) => {
      const identifier = getState()[items[0].id].identifier;
      const headers = { ...options.destination.headers, "x-upload-identifier": identifier };

            return {
                options: { destination: {  headers  } },
            };
        });

Could this work? Is it missing something?

AFAIK many apps already have some kind of mapping between upload items and their internal state for all sorts of reasons (ie: progress tracking) so this kind of addition doesn't change much in the overall sense.

[mwdiaz]

Yup, this works and is more or less what we're already doing for other data (so as you point out, including additional data doesn't change much overall). I totally understand your stance of trying to keep this sort of implementation detail out of the core and leaving it up to consumers. I figured I'd ask since it seemed that the existing userData prop was a step in that direction but it doesn't sound like you're super thrilled with that already 😄.

[yoavniran]

Thanks Mike.
I guess you could use the userData as your state and add "slices" to it based on the item id. It will be up to you to manage but it will be part of the batch data rather than outside Uploady.

In any case, thanks for understanding and I'll keep an eye for future requests and whether a strong need comes to implement this inside Uploady.