You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the DataBuffer receives transform and target_transform and applies them when accessing an item. When deserializing the Learner state, this requires resetting the transforms (see DataBuffer.set_transforms) and generally bloats the code of the buffer.
I would propose to remove the transforms form the buffer and instead generalize the _TransformedDataset such that it can be applied to DataBuffers as well. This way, transforms are handled in a unified way by wrapping all relevant datasets in _TransformedDataset in the Learner.
To make _TransformedDataset more general, instead of passing transform and target_transform, we could pass a single transform that operates on the entire data point. When wrapping a buffer in _TransformedDataset, we just need to make sure that the metadata are "ignored" (e.g., transform=lambda (data_point, metadata): (my_transform(data_point), metadata)).
The text was updated successfully, but these errors were encountered:
Currently, the
DataBuffer
receivestransform
andtarget_transform
and applies them when accessing an item. When deserializing theLearner
state, this requires resetting the transforms (seeDataBuffer.set_transforms
) and generally bloats the code of the buffer.I would propose to remove the transforms form the buffer and instead generalize the
_TransformedDataset
such that it can be applied toDataBuffer
s as well. This way, transforms are handled in a unified way by wrapping all relevant datasets in_TransformedDataset
in theLearner
.To make
_TransformedDataset
more general, instead of passingtransform
andtarget_transform
, we could pass a single transform that operates on the entire data point. When wrapping a buffer in_TransformedDataset
, we just need to make sure that the metadata are "ignored" (e.g.,transform=lambda (data_point, metadata): (my_transform(data_point), metadata)
).The text was updated successfully, but these errors were encountered: