• Adam Simpkins's avatar
    change the mechanism for "internal" buffer storage · cb527272
    Adam Simpkins authored
    Summary:
    This removes kFlagExt, and instead implements the internal buffer
    optimization using operator new and delete.
    
    IOBuf::createCombined() will allocate a single buffer for the IOBuf,
    SharedInfo, and the actual data itself.  Each heap allocated IOBuf now
    contains a set of flags indicating if the memory segment indicating when
    it can actually be deleted.  The delete operator won't actually free the
    storage until both the IOBuf and the data buffer are unused (the
    SharedInfo object always becomes unused at the same time as the data
    buffer).
    
    This has a couple advantages over the old mechanism:
    
    - Other IOBufs can continue to use and share the data storage space even
    after the original IOBuf associated with it is deleted.  With the old
    internal buffer mechanism, internal buffers could never be shared.
    
    - This simplifies several parts of the code, since kFlagExt no longer
    exists.  Code that previously required special handling for internal
    buffers no longer needs to be aware of them.
    
    One downside is that this can result in wasted space in some cases if
    the original IOBuf is changed to point at a different buffer, since the
    space for the data buffer cannot be freed until the IOBuf itself is also
    destroyed.  The old internal buffer mechanism also had this problem,
    which we mitigated simply by disallowing internal buffers for larger
    than ~200 bytes.  With the new mechanism we currently allocate an
    internal buffer for up to 1024 bytes by default, but we also allow
    callers to explicitly decide if they want an internal buffer or not.
    
    Test Plan:
    Added some new unit tests for the combined buffer behavior.  Also ran
    all of the existing IOBuf unit tests, with and without ASAN.  (ASAN
    performs additional memory checking, but also changes IOBuf's behavior
    slightly as usingJEMalloc() is false with ASAN.)
    
    Reviewed By: davejwatson@fb.com
    
    FB internal diff: D1072336
    
    @override-unit-failures
    cb527272
IOBuf.cpp 24.8 KB