Skip to content

Commit

Permalink
Replace boost::shared_ptr with std::shared_ptr (#168)
Browse files Browse the repository at this point in the history 
Boost version 1.63 and newer [supports](boostorg/python#29) `std::shared_ptr` python converters.  Thus, in this PR, all boost shared pointers are replaced with standard library shared pointers everywhere in the library.  The required C++ standard is also increased to C++17, since shared pointers for allocated C arrays do not work with the older standards.

In the process, this PR also updates the cereal library to the latest version (1.3.2) without any patching.
  • Loading branch information
@arahlin
arahlin authored Nov 20, 2024
1 parent efe5a39 commit 88abb76
Show file tree
Hide file tree
Showing 87 changed files with 494 additions and 372 deletions.
7 changes: 2 additions & 5 deletions CMakeLists.txt
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ set(CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} $ENV{SITE_CMAKE_DIR} "${CMAKE_SOURCE_
# Work around stupid broken Red Hat systems
set(CMAKE_PLATFORM_IMPLICIT_LINK_DIRECTORIES "")

# Require C++ 14
set(CMAKE_CXX_STANDARD 14)
# Require C++ 17
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

Expand Down Expand Up @@ -86,9 +86,6 @@ target_compile_definitions(spt3g INTERFACE -DBOOST_PYTHON_MAX_ARITY=20
if(Boost_VERSION VERSION_LESS 1.77)
target_compile_definitions(spt3g INTERFACE -DBOOST_BIND_GLOBAL_PLACEHOLDERS)
endif()
if(Boost_VERSION VERSION_LESS 1.82)
target_compile_definitions(spt3g INTERFACE -DBOOST_NO_CXX11_SMART_PTR=1)
endif()

# Shell script to set environment variables
configure_file(${CMAKE_SOURCE_DIR}/cmake/env-shell.sh.in ${CMAKE_BINARY_DIR}/env-shell.sh @ONLY)
Expand Down
2 changes: 1 addition & 1 deletion README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ This depends on Boost and cmake, as well as the usual Python packages. Some addi
Minimum versions:

- GCC >= 5.0 or clang >= 3.4
- Boost >= 1.48
- Boost >= 1.63
- cmake >= 3.12
- Python >= 2.7 (although pre-Python-3 support is best-effort)

Expand Down
7 changes: 1 addition & 6 deletions core/include/cereal/README
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1 @@
This is the cereal library, as of 11/4/2019. The following patches have been applied:
- All std::shared_ptr and friends replaced by boost::shared_ptr and friends using sed for compatibility with boost::python.
- Instances of nullptr removed when initializing boost::shared_ptr for compatibility with ancient boost versions

Sed script:
sed -E -I '' -e 's/std::shared_ptr/boost::shared_ptr/g' -e 's/std::static_pointer_cast/boost::static_pointer_cast/g' -e 's/std::dynamic_pointer_cast/boost::dynamic_pointer_cast/g' -e 's/std::make_shared/boost::make_shared/g'
This is the cereal library, version 1.3.2. No patches have been applied.
4 changes: 2 additions & 2 deletions core/include/cereal/access.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down
4 changes: 2 additions & 2 deletions core/include/cereal/archives/adapters.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down
4 changes: 2 additions & 2 deletions core/include/cereal/archives/binary.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down
35 changes: 20 additions & 15 deletions core/include/cereal/archives/json.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down Expand Up @@ -286,19 +286,19 @@ namespace cereal
void saveLong(T lu){ saveValue( static_cast<std::uint64_t>( lu ) ); }

public:
#ifdef _MSC_VER
#if defined(_MSC_VER) && _MSC_VER < 1916
//! MSVC only long overload to current node
void saveValue( unsigned long lu ){ saveLong( lu ); };
#else // _MSC_VER
//! Serialize a long if it would not be caught otherwise
template <class T, traits::EnableIf<std::is_same<T, long>::value,
!std::is_same<T, std::int32_t>::value,
!std::is_same<T, int>::value,
!std::is_same<T, std::int64_t>::value> = traits::sfinae> inline
void saveValue( T t ){ saveLong( t ); }

//! Serialize an unsigned long if it would not be caught otherwise
template <class T, traits::EnableIf<std::is_same<T, unsigned long>::value,
!std::is_same<T, std::uint32_t>::value,
!std::is_same<T, unsigned>::value,
!std::is_same<T, std::uint64_t>::value> = traits::sfinae> inline
void saveValue( T t ){ saveLong( t ); }
#endif // _MSC_VER
Expand Down Expand Up @@ -483,16 +483,16 @@ namespace cereal
Iterator() : itsIndex( 0 ), itsType(Null_) {}

Iterator(MemberIterator begin, MemberIterator end) :
itsMemberItBegin(begin), itsMemberItEnd(end), itsIndex(0), itsType(Member)
itsMemberItBegin(begin), itsMemberItEnd(end), itsIndex(0), itsSize(std::distance(begin, end)), itsType(Member)
{
if( std::distance( begin, end ) == 0 )
if( itsSize == 0 )
itsType = Null_;
}

Iterator(ValueIterator begin, ValueIterator end) :
itsValueItBegin(begin), itsIndex(0), itsType(Value)
itsValueItBegin(begin), itsIndex(0), itsSize(std::distance(begin, end)), itsType(Value)
{
if( std::distance( begin, end ) == 0 )
if( itsSize == 0 )
itsType = Null_;
}

Expand All @@ -506,6 +506,9 @@ namespace cereal
//! Get the value of the current node
GenericValue const & value()
{
if( itsIndex >= itsSize )
throw cereal::Exception("No more objects in input");

switch(itsType)
{
case Value : return itsValueItBegin[itsIndex];
Expand Down Expand Up @@ -546,7 +549,7 @@ namespace cereal
private:
MemberIterator itsMemberItBegin, itsMemberItEnd; //!< The member iterator (object)
ValueIterator itsValueItBegin; //!< The value iterator (array)
size_t itsIndex; //!< The current index of this iterator
size_t itsIndex, itsSize; //!< The current index of this iterator
enum Type {Value, Member, Null_} itsType; //!< Whether this holds values (array) or members (objects) or nothing
};

Expand All @@ -561,18 +564,20 @@ namespace cereal
@throws Exception if an expectedName is given and not found */
inline void search()
{
// store pointer to itsNextName locally and reset to nullptr in case search() throws
auto localNextName = itsNextName;
itsNextName = nullptr;

// The name an NVP provided with setNextName()
if( itsNextName )
if( localNextName )
{
// The actual name of the current node
auto const actualName = itsIteratorStack.back().name();

// Do a search if we don't see a name coming up, or if the names don't match
if( !actualName || std::strcmp( itsNextName, actualName ) != 0 )
itsIteratorStack.back().search( itsNextName );
if( !actualName || std::strcmp( localNextName, actualName ) != 0 )
itsIteratorStack.back().search( localNextName );
}

itsNextName = nullptr;
}

public:
Expand Down
4 changes: 2 additions & 2 deletions core/include/cereal/archives/portable_binary.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down
4 changes: 2 additions & 2 deletions core/include/cereal/archives/xml.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down
53 changes: 42 additions & 11 deletions core/include/cereal/cereal.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
* Neither the name of cereal nor the
* Neither the name of the copyright holder nor the
names of its contributors may be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL RANDOLPH VOORHIES OR SHANE GRANT BE LIABLE FOR ANY
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
Expand Down Expand Up @@ -112,7 +112,7 @@ namespace cereal
@code{.cpp}
struct MyEdge
{
boost::shared_ptr<MyNode> connection;
std::shared_ptr<MyNode> connection;
int some_value;
template<class Archive>
Expand Down Expand Up @@ -258,6 +258,26 @@ namespace cereal
Interfaces for other forms of serialization functions is similar. This
macro should be placed at global scope.
@ingroup Utility */

//! On C++17, define the StaticObject as inline to merge the definitions across TUs
//! This prevents multiple definition errors when this macro appears in a header file
//! included in multiple TUs.
#ifdef CEREAL_HAS_CPP17
#define CEREAL_CLASS_VERSION(TYPE, VERSION_NUMBER) \
namespace cereal { namespace detail { \
template <> struct Version<TYPE> \
{ \
static std::uint32_t registerVersion() \
{ \
::cereal::detail::StaticObject<Versions>::getInstance().mapping.emplace( \
std::type_index(typeid(TYPE)).hash_code(), VERSION_NUMBER ); \
return VERSION_NUMBER; \
} \
static inline const std::uint32_t version = registerVersion(); \
CEREAL_UNUSED_FUNCTION \
}; /* end Version */ \
} } // end namespaces
#else
#define CEREAL_CLASS_VERSION(TYPE, VERSION_NUMBER) \
namespace cereal { namespace detail { \
template <> struct Version<TYPE> \
Expand All @@ -275,6 +295,8 @@ namespace cereal
Version<TYPE>::registerVersion(); \
} } // end namespaces

#endif

// ######################################################################
//! The base output archive class
/*! This is the base output archive for all output archives. If you create
Expand Down Expand Up @@ -369,12 +391,17 @@ namespace cereal
point to the same data.
@internal
@param addr The address (see shared_ptr get()) pointed to by the shared pointer
@param sharedPointer The shared pointer itself (the adress is taked via get()).
The archive takes a copy to prevent the memory location to be freed
as long as the address is used as id. This is needed to prevent CVE-2020-11105.
@return A key that uniquely identifies the pointer */
inline std::uint32_t registerSharedPointer( void const * addr )
inline std::uint32_t registerSharedPointer(const std::shared_ptr<const void>& sharedPointer)
{
void const * addr = sharedPointer.get();

// Handle null pointers by just returning 0
if(addr == 0) return 0;
itsSharedPointerStorage.push_back(sharedPointer);

auto id = itsSharedPointerMap.find( addr );
if( id == itsSharedPointerMap.end() )
Expand Down Expand Up @@ -454,7 +481,7 @@ namespace cereal
template <class T> inline
ArchiveType & processImpl(DeferredData<T> const & d)
{
std::function<void(void)> deferment( [=](){ self->process( d.value ); } );
std::function<void(void)> deferment( [this, d](){ self->process( d.value ); } );
itsDeferments.emplace_back( std::move(deferment) );

return *self;
Expand Down Expand Up @@ -645,6 +672,10 @@ namespace cereal
//! Maps from addresses to pointer ids
std::unordered_map<void const *, std::uint32_t> itsSharedPointerMap;

//! Copy of shared pointers used in #itsSharedPointerMap to make sure they are kept alive
// during lifetime of itsSharedPointerMap to prevent CVE-2020-11105.
std::vector<std::shared_ptr<const void>> itsSharedPointerStorage;

//! The id to be given to the next pointer
std::uint32_t itsCurrentPointerId;

Expand Down Expand Up @@ -759,9 +790,9 @@ namespace cereal
@param id The unique id that was serialized for the pointer
@return A shared pointer to the data
@throw Exception if the id does not exist */
inline boost::shared_ptr<void> getSharedPointer(std::uint32_t const id)
inline std::shared_ptr<void> getSharedPointer(std::uint32_t const id)
{
if(id == 0) return boost::shared_ptr<void>(nullptr);
if(id == 0) return std::shared_ptr<void>(nullptr);

auto iter = itsSharedPointerMap.find( id );
if(iter == itsSharedPointerMap.end())
Expand All @@ -777,7 +808,7 @@ namespace cereal
@internal
@param id The unique identifier for the shared pointer
@param ptr The actual shared pointer */
inline void registerSharedPointer(std::uint32_t const id, boost::shared_ptr<void> ptr)
inline void registerSharedPointer(std::uint32_t const id, std::shared_ptr<void> ptr)
{
std::uint32_t const stripped_id = id & ~detail::msb_32bit;
itsSharedPointerMap[stripped_id] = ptr;
Expand Down Expand Up @@ -859,7 +890,7 @@ namespace cereal
template <class T> inline
ArchiveType & processImpl(DeferredData<T> const & d)
{
std::function<void(void)> deferment( [=](){ self->process( d.value ); } );
std::function<void(void)> deferment( [this, d](){ self->process( d.value ); } );
itsDeferments.emplace_back( std::move(deferment) );

return *self;
Expand Down Expand Up @@ -1073,7 +1104,7 @@ namespace cereal
std::unordered_set<traits::detail::base_class_id, traits::detail::base_class_id_hash> itsBaseClassSet;

//! Maps from pointer ids to metadata
std::unordered_map<std::uint32_t, boost::shared_ptr<void>> itsSharedPointerMap;
std::unordered_map<std::uint32_t, std::shared_ptr<void>> itsSharedPointerMap;

//! Maps from name ids to names
std::unordered_map<std::uint32_t, std::string> itsPolymorphicTypeMap;
Expand Down
Loading

0 comments on commit 88abb76

Please sign in to comment.