Tuesday, July 1, 2008

asio and asynchronous file processing

Some weeks ago I began tests of reading amount numbers of floats from a text file. I wrote simple test framework and test and compare different methods of file reading. Alex advises me to try the asio in my tests. He advises me to try a version from CVS but I decide to try the latest version – 1.1.0. I use Windows and all my experiments related to it.

Asio has class asio::windows::basic_stream_handle that allows to read stream of data from file. But in the latest version this class has a bug – file pointer not moved and all subsequent calls read data from begin of the file.

The last version from CVS has class asio::windows::basic_random_access_handle that allows to read block (or stream) of data from arbitrary position in file. Using this class I can make my tests.

Test file has following structure:
float float float float
float float float float

float float float float

float float float float

How we read data from the file using the asio? Asio provide functions and classes for read data in two modes – a synchronous and asynchronous.

In my tests I use a hybrid scheme – I read data synchronously for locate section in file and use the asynchronous mode and double-buffering idiom for process float list. Also I run tests only on the first section so code for location section has a small bug.

And now super prize for patient readers – source code of my tests, full source (sorry, for filefactory).

And so, first I define reading handler that will be invoked by the asio kernel when read data will be ready to process. In my sources it's a read_handler function.

Second, before start read data we should define condition with reading will stop. In my sources it's a complete_condition function.

Third, we should define destination of the read data. In the most examples boost::array used or asio::streambuf. boost::array I can't use because reading data without buffering requires that data buffer should be sector aligned. asio::streambuf has its own disadvantages. And so, I use raw pointers.

As I wrote above I early had written realization of test using standard Windows API. I compare results that were obtained and I can conclude that a realization of the asio library is a very and very good.

test of random access from file without buffering (FILE_FLAG_NO_BUFFERING)
On the horizontal bar - size of data buffer (bytes), on the vertical bar - data transfer rate (MB/s)

And also I will choose the asio for all my following projects that require file processing first.

No comments: