2010-11-21

Reading a memory mapped file as a normal C++ std::istream.

I was working on an entry to a programming challenge that was all about reading numbers from an file as fast as possible. In that work, I did some experimentation with specializing a std::streambuf object to be able to read from a memory map of that file as if it was a normal C++ input stream (std::istream).

It turned out that it was not the fastest way to use the memory map since the istream machinery generated by my C++ compiler was not that very efficient compared to parsing the memory more directly. Nevertheless, I thought it was a neat solution so I would like to share the code with the world in case someone else needs it.

Here it is:

/// Streambuf specialization to read (only) from
/// a file using a memory map of the file.
/// This is Microsoft Windows specific but could
/// relatively easy be ported to a Linux variant.
/// Also, the whole file is mapped to memory. There
/// is room for optimization with reading only one
/// page at the time...
///
/// Usage example to read all lines from a file:
///
///     membuf mb("a_file.txt");
///     std::istream i(&mb);
///     for( std::string line; std::getline(i,line); )
///     {
///        // do something with the line.
///     }
///
/// Author: Peter Jansson (http://www.p-jansson.com)
/// Date: 2010-11-21
/// Dedicated to the public domain.
///
#include "streambuf"
#include "windows.h"
class membuf: public std::streambuf
{
   public:
      membuf(const char* filename)
         :std::streambuf()
      {
         hFile = CreateFileA(
               filename,
               GENERIC_READ,
               0,
               NULL,
               OPEN_EXISTING,
               FILE_ATTRIBUTE_READONLY,
               NULL);
         hFileMappingObject = CreateFileMapping(
               hFile,
               NULL,
               PAGE_READONLY,
               0,
               0,
               NULL);
         beg = MapViewOfFile(hFileMappingObject,
               FILE_MAP_READ,
               0,
               0,
               0);
         char* e = (char*)beg;
         e += GetFileSize( hFile, NULL );
         setg((char*)beg,(char*)beg,e);
         setp(&outBuf,&outBuf);
      }
      /// Close the memory mapping and the opened
      /// file.
      ~membuf()
      {
         UnmapViewOfFile(beg);
         CloseHandle( hFileMappingObject );
         CloseHandle( hFile );
      }
   private:
      /// We can't copy this.
      membuf(const membuf&);
      membuf& operator=(const membuf&);
      /// Outbuf buffer that is not used since we
      /// don't write to the memory map.
      char outBuf;
      HANDLE hFile,hFileMappingObject;
      /// Pointer to the beginning of the mapped
      /// memory area.
      LPVOID beg;
};

0 kommentarer:

Post a Comment