vectorflow.dataset

This module provides utility classes to iterate over data.

It is not mandatory to use them when using vectorflow, but you might find them useful and slightly more intuitive to use than the built-in range mechanism if you're a beginner with D.

When creating a dataset for vectorflow, it is important for the data sharding to be thread-safe if learning over multiple cores is considered, as data parallelism with Hogwild is the main strategy used. By default, vectorflow will try to shard the forward range provided with std.range.evenChunks, which might or might not work depending on your specific reader. To explicitly shard the data, just specify an evenChunks function in your reader implementation (see MultiFilesReader for an example).

Members

Classes

DataReader
class DataReader(T)
MultiFilesReader
class MultiFilesReader(T)

Meta