Kamaelia.File.MaxSpeedFileReader

• component MaxSpeedFileReader

• Reading a file as fast as possible
  • Example Usage
  • More details

Reading a file as fast as possible

MaxSpeedFileReader reads a file in bytes mode as fast as it can; limited only by any size limit on the inbox it is sending the data to.

This component is therefore useful for building systems that are self rate limiting - systems that are just trying to process data as fast as they can and are limited by the speed of the slowest part of the chain.

Example Usage

Read "myfile" in in chunks of 1024 bytes. The rate is limited by the rate at which the consumer component can consume the chunks, since its inbox has a size limit of 5 items of data:

consumer = Consumer()
consumer.inboxes["inbox"].setSize(5)

Pipeline( MaxSpeedFileReader("myfile", chunksize=1024),
          consumer,
        ).run()

More details

Specify a filename and chunksize and MaxSpeedFileReader will read bytes from the file in the chunksize you specified and send them out of its "outbox" outbox.

If the destination inbox it is sending chunks to is size limited, then MaxSpeedFileReader will pause until space becomes available. This is how the speed at which the file is ingested is regulated - by the rate at which it is consumed.

When the whole file has been read, this component will terminate and send a producerFinished() message out of its "signal" outbox.

If a producerFinished message is received on the "control" inbox, this component will complete sending any data that may be waiting. It will then send the producerFinished message on out of its "signal" outbox and terminate.

If a shutdownMicroprocess message is received on the "control" inbox, this component will immediately send it on out of its "signal" outbox and immediately terminate. It will not complete sending on any pending data.

---

Kamaelia.File.MaxSpeedFileReader.MaxSpeedFileReader

class MaxSpeedFileReader(Axon.Component.component)

MaxSpeedFileReader(filename[,chunksize]) -> new MaxSpeedFileReader component.

Reads the contents of a file in bytes mode; sending it out as fast as it can in chunks from the "outbox" outbox. The rate of reading is only limited by any size limit of the destination inbox to which the data is being sent.

Keyword arguments:

• filename -- The filename of the file to read
• chunksize -- Optional. The maximum number of bytes in each chunk of data read from the file and sent out of the "outbox" outbox (default=32768)

Inboxes

Outboxes

Methods defined here

__init__(self, filename[, chunksize])

canStop(self)

handleControl(self)

main(self)

mustStop(self)

waitSend(self, data, boxname)