From: http://www.mr-edd.co.uk/blog/beginners_guide_streambuf
A beginner's guide to writing a custom stream buffer (std::streambuf
)
Streams are one of the major abstractions provided by the STL as part of the C++ standard library. Every newcomer to C++ is taught to write "Hello,
to the console using
world!"std::cout
,
which is itself an std::ostream
object
and std::cin
an std::istream
object.
There's a lot more to streams than cout
and cin
,
however. In this post I'll look at how we can extend C++ streams by creating our own custom stream buffers. Note that beginner
in
the title of this post refers to someone that's never implemented a custom stream buffer before and not necessarily a beginner to C++ in general; you will need at least a basic knowledge of how C++ works in order to follow this post. The code for all the examples
is available at the end.
The C++ standard library provides the primary interface to manipulating the contents of files on disk through the std::ofstream
, std::ifstream
and std::fstream
classes.
We also havestringstreams
,
which allow you to treat strings as streams and therefore compose a string from the textual representations of various types.
std::ostringstream oss;
oss << "Hello, world!\\n";
oss << 123 << '\\n';
std::string s = oss.str();
Similarly, we're able to read data from a string by employing an std::istringstream
and
using the natural extraction (>>
)
operator.
Boost's lexical_cast
facility
uses this to good effect to allow conversions between types whose text representations are compatible, as well as a simple facility for quickly getting a string representation for an object of an 'output streamable type'.
using boost::lexical_cast;
using std::string;
int x = 5;
string s = lexical_cast<string>(x);
assert(s == "5");
At the heart of this flexibility is the stream
buffer, which deals with the buffering and transportation of characters to or from their target or source, be it a file, a string, the system console or something else entirely. We could stream text over a network connection or from the flash memory
of a particular device all through the same interface. The stream buffer is defined in a way that is orthogonal to the stream that is using it and so we are often able to swap and change the buffer a given stream is using at any given moment to redirect output
elsewhere, if we so desire. I guess C++ streams are an example of the strategy
design pattern in this respect.
For instance, we can edit the standard logging stream (std::clog
)
to write in to a string stream, rather than its usual target, by making it use the string stream's buffer:
#include <iostream>
#include <iomanip>
#include <string>
#include <sstream>
int main()
{
std::ostringstream oss;
// Make clog use the buffer from oss
std::streambuf *former_buff =
std::clog.rdbuf(oss.rdbuf());
std::clog << "This will appear in oss!" << std::flush;
std::cout << oss.str() << '\\n';
// Give clog back its previous buffer
std::clog.rdbuf(former_buff);
return 0;
}
However, creating your own stream buffer can be a little tricky, or at least a little intimidating when you first set out to do so. So the idea of this post is to provide some example implementations for a number of useful stream buffers as a platform for discussion.
Let's first look at some of the underlying concepts behind a stream buffer. All stream buffers are derived from the std::streambuf
base
class, whose virtual functions we must override in order to implement the customised behaviour of our particular stream buffer type. An std::streambuf
is
an abstraction of an array of chars that has its data sourced from or sent to a sequential access device. Under certain conditions the array will be re-filled (for an input buffer) or flushed and
emptied (for an output buffer).
When inserting data in to an ostream
(using <<
,
for example), data is written in to the buffer's array. When this array overflows,
the data in the array is flushed to the destination (or sink)
and the state associated with the array is reset, ready for more characters.
When extracting data from an istream
(using >>
,
for example), data is read from the buffer's array. When there is no more data left to read, that is, when the array underflows,
the contents of the array are re-filled with data from the source and the state associated with the array is reset.
To keep track of the different areas in the stream buffer arrays, six pointers are maintained internally, three for input and three for output.
For an output stream buffer, there are:
-
the put
base pointer, as returned fromstd::streambuf::pbase()
,
which points to the first element of the buffer's internal array, -
the put
pointer, as returned fromstd::streambuf::pptr()
,
which points to the next character in the buffer that may be written to -
and the end
put pointer as returned fromstd::streambuf::epptr()
,
which points to one-past-the-last-element of the buffer's internal array.
Typically, pbase()
and epptr()
won't
change at all; it will only be pptr()
that
changes as the buffer is used.
For an input stream buffer, we have 3 different pointers to contend with, though they have a roughly analogous purpose. We have:
-
the end
back pointer, as returned fromstd::streambuf::eback()
,
which points to the last character (lowest in address) in the buffer's internal array in to which a character may be put
back, -
the get
pointer, as returned fromstd::streambuf::gptr()
,
which points to the character in the buffer that will be extracted next
by theistream
-
and the end
get pointer, as returned fromstd::streambuf::egptr()
,
which points to one-past-the-last-element of the buffer's internal array.
Again, it is typically the case that eback()
and egptr()
won't
change during the life-time of thestreambuf
.
Input stream buffers, written for use with istreams
,
tend to be a little bit more complex than output buffers, written for ostreams
.
This is because we should endeavor to allow the user to put characters back in to the stream, to a reasonable degree, which is done through the std::istream
'sputback()
member
function. What this means is that we need to maintain a section at the start of the array for put-back space.
Typically, one character of put-back space is expected, though there's no reason we shouldn't be able to provide more, in general.
Now you may have noticed that we are deriving from std::streambuf
in
order to create both an output buffer and an input buffer; there is no std::istreambuf
or std::ostreambuf
.
This is because it is possible to provide a stream buffer that manipulates the same internal array as a buffer for both reading from and writing to an external entity. This is what std::fstream
does,
for example. However, implementing a dual-purpose streambuf
is
a fair bit trickier, so I won't be considering it in this post.
It is also possible to create buffers for wide character streams. std::streambuf
is
actually atypedef
for std::basic_streambuf<char>
.
Similarly there exists std::wstreambuf
,
a