alibaba yalantingLibs struct_pack代码梳理

struct_pack 接口

支持序列化所有的STL容器、自定义容器和optional
对于自定义的类,目前只能序列化std::is_aggregate_v是true的类,也就是聚合.
不支持指针等的序列化,好评是支持代表所有权的unique_ptr序列化,合理
说实话,功能目前还是很简略,肯定是不如boost或者protobuf的,但是代码里面还是有很多trick的,适合填充c++模板入门到初学阶段的代码阅读量.但是想写出这样的代码,估计得c++模板良到熟练了.
注:看template代码真的折寿

序列化

struct person {int64_t id;std::string name;int age;double salary;
};

序列化对象到新字节容器

std::vector<char> result = struct_pack::serialize(person1);

序列化对象到容器尾部

std::string result="The next line is struct_pack serialize result.\n";
auto result = struct_pack::serialize_to(result,person1);

将序列化结果保存到指针指向的内存中

auto sz=struct_pack::get_needed_siarray(person1);
std::unique_ptr array=std::make_unique<char[]>(sz);
auto result = struct_pack::serialize_to(array.get(),sz,person1);

多参数序列化

序列化为一个tuple 然后搭配元组展开

auto result=struct_pack::serialize(person1.id, person1.name, person1.age, person1.salary);
//serialize as std::tuple<int64_t, std::string, int, double>

将序列化结果保存到输出流

std::ofstream writer("struct_pack_demo.data",std::ofstream::out | std::ofstream::binary);
struct_pack::serialize_to(writer, person1);

自定义类型序列化

// We should not inherit from stl container, this case just for testing.
template <typename Key, typename Value>
struct my_map : public std::map<Key, Value> {};my_map<int, std::string> map1;
map1.emplace(1, "tom");
map1.emplace(2, "jerry");absl::flat_hash_map<int, std::string> map2 ={{1, "huey"}, {2, "dewey"}, {3, "louie"},};auto buffer1 = serialize(map1);
auto buffer2 = serialize(map2);

序列化到自定义的输出流

该流需要满足以下约束条件：

template <typename T>
concept writer_t = requires(T t) {t.write((const char *)nullptr, std::size_t{}); //向流输出一段数据。返回值应能隐式转换为bool值，出错时应返回false。
};

例如:

//一个简单的输出流，对fwrite函数进行封装。
struct fwrite_stream {FILE* file;bool write(const char* data, std::size_t sz) {return fwrite(data, sz, 1, file) == 1;}fwrite_stream(const char* file_name) : file(fopen(file_name, "wb")) {}~fwrite_stream() { fclose(file); }
};
// ...
fwrite_stream writer("struct_pack_demo.data");
struct_pack::serialize_to(writer, person);

反序列化

基本反序列化

auto person2 = deserialize<person>(buffer);
assert(person2); // person2.has_value() == true
assert(person2.value()==person1);

从指针指向的内存中反序列化

auto person2 = deserialize<person>(buffer.data(),buffer.size());
assert(person2); //person2.has_value() == true
assert(person2.value()==person1);

反序列化到已有对象

person person2;
std::errc ec = deserialize_to(person2, buffer);
assert(ec==std::errc{}); // person2.has_value() == true
assert(person2==person1);

多参数反序列化

auto person2 = deserialize<int64_t,std::string,int,double>(buffer);
assert(person2); // person2.has_value() == true
auto &&[id,name,age,salary]=person2.value();
assert(person1.id==id);
assert(person1.name==name);
assert(person1.age==age);
assert(person1.salary==salary);

输入流反序列化

std::ifstream ifs("struct_pack_demo.data",std::ofstream::in | std::ofstream::binary);
auto person2 = struct_pack::deserialize<person>(ifs);
assert(person2 == person1);

部分反序列化

// 只反序列化person的第2个字段
auto name = get_field<person, 1>(buffer.data(), buffer.size());
assert(name); // name.has_value() == true
assert(name.value() == "hello struct pack");

从自定义的输入流中反序列化

该流需要满足以下约束条件：

template <typename T>
concept reader_t = requires(T t) {t.read((char *)nullptr, std::size_t{}); //从流中读取一段数据。返回值应能隐式转换为bool值，出错时应返回false。t.ignore(std::size_t{}); //从流中跳过一段数据。返回值应能隐式转换为bool值，出错时应返回false。t.tellg(); //返回一个无符号整数，代表当前的绝对读取位置
};

此外，如果该流还额外支持read_view函数，则支持对string_view的零拷贝优化。

template <typename T>
concept view_reader_t = reader_t<T> && requires(T t) {{ t.read_view(std::size_t{}) } -> std::convertible_to<const char *>;//从流中读取一段视图（零拷贝读取），返回值为该视图指向的起始位置，出错时应返回空指针。
};

示例:

//一个简单的输入流，对fread函数进行封装。
struct fread_stream {FILE* file;bool read(char* data, std::size_t sz) {return fread(data, sz, 1, file) == 1;}bool ignore(std::size_t sz) { return fseek(file, sz, SEEK_CUR) == 0; }std::size_t tellg() {//if you worry about ftell performance, just use an variable to record it.return ftell(file);}fread_stream(const char* file_name) : file(fopen(file_name, "rb")) {}~fread_stream() { fclose(file); }
};//...fread_stream ifs("struct_pack_demo.data");
auto person2 = struct_pack::deserialize<person>(ifs);
assert(person2 == person);

struct_pack 代码分析

从示例开始 最简单的序列化

从一个最基本的序列化代码开始

struct person {int64_t id;std::string name;int age;double salary;
};person person1{.id = 1, .name = "hello struct pack", .age = 20, .salary = 1024.42};
std::vector<char> result = struct_pack::serialize(person1);

函数调用

struct_pack::serialize(person1);

模板类:

1. buffer 默认是std::vector<char>
2. serialize_config conf 配置 结构体封装的enum class 有三种类型{ automatic, disable, enable };默认是automatic
3. typename... Args变长模板参数:读取传入的参数

修饰:

1. [[nodiscard]] 返回值不可丢弃
2. #define STRUCT_PACK_INLINE __attribute__((always_inline)) inline 总是内联

参数:

1. 需要序列化的参数的常引用const Args &...args

函数主体:

template <detail::struct_pack_buffer Buffer = std::vector<char>,serialize_config conf = serialize_config{}, typename... Args>
[[nodiscard]] STRUCT_PACK_INLINE Buffer serialize(const Args &...args) {static_assert(sizeof...(args) > 0);//变长参数需要>0Buffer buffer;//返回的bufferserialize_to<conf>(buffer, args...);//调用更广的处理函数,即在buffer后添加 return buffer;
}

serialize_to(Buffer &buffer, const Args &…args)

模板类:

1. serialize_config conf 配置 结构体封装的enum class 有三种类型{ automatic, disable, enable };默认是automatic
2. detail::struct_pack_buffer Buffer 符合约束条件的buffer 有两个concept trivially_copyable_container:一个是对容器的约束:
   连续容器continuous_container

//判断是否为连续容器template <typename Type>concept continuous_container =string<Type> || (container<Type> && requires(Type container) {std::span{container};});

是否可以序列化is_trivial_serializable

  template <typename T, bool ignore_compatible_field = false>struct is_trivial_serializable {private:static constexpr bool solve() {if constexpr (is_compatible_v<T> || trivial_view<T>) {return ignore_compatible_field;}else if constexpr (std::is_enum_v<T> || std::is_fundamental_v<T>) {return true;}else if constexpr (array<T>) {return is_trivial_serializable<typename T::value_type,ignore_compatible_field>::value;}else if constexpr (c_array<T>) {return is_trivial_serializable<typename std::remove_all_extents<T>::type,ignore_compatible_field>::value;}else if constexpr (!pair<T> && tuple<T> && !is_trivial_tuple<T>) {return false;}else if constexpr (container<T> || optional<T> || variant<T> ||unique_ptr<T> || expected<T> || container_adapter<T> ||varint_t<T>) {return false;}else if constexpr (pair<T>) {return is_trivial_serializable<typename T::first_type,ignore_compatible_field>::value &&is_trivial_serializable<typename T::second_type,ignore_compatible_field>::value;}else if constexpr (is_trivial_tuple<T>) {return []<std::size_t... I>(std::index_sequence<I...>)CONSTEXPR_INLINE_LAMBDA {return (is_trivial_serializable<std::tuple_element_t<I, T>,ignore_compatible_field>::value &&...);}(std::make_index_sequence<std::tuple_size_v<T>>{});}else if constexpr (std::is_class_v<T>) {using T_ = decltype(get_types<T>());return []<std::size_t... I>(std::index_sequence<I...>)CONSTEXPR_INLINE_LAMBDA {return (is_trivial_serializable<std::tuple_element_t<I, T_>,ignore_compatible_field>::value &&...);}(std::make_index_sequence<std::tuple_size_v<T_>>{});}elsereturn false;}public:static inline constexpr bool value = is_trivial_serializable::solve();};

一个是对容器内容物的约束,第二个从concept,容器的内容是否是需要的byte类型 只接受三种 char,unsigned char, std::byte.

template <typename T>
concept struct_pack_byte = std::is_same_v<char, T>|| std::is_same_v<unsigned char, T>|| std::is_same_v<std::byte, T>;

4. typename... Args变长模板参数:读取传入的参数

修饰:

1. [[nodiscard]] 返回值不可丢弃
2. #define STRUCT_PACK_INLINE __attribute__((always_inline)) inline 总是内联

参数:

1. buffer需要附加到尾部的buffer
2. 需要序列化的参数的常引用const Args &...args

函数主体:

template <serialize_config conf = serialize_config{},detail::struct_pack_buffer Buffer, typename... Args>
void STRUCT_PACK_INLINE serialize_to(Buffer &buffer, const Args &...args) {
//输入需要附加的容器和需要序列化的内容static_assert(sizeof...(args) > 0);//合法性判断auto data_offset = buffer.size();//得到offset,即从哪个index开始填auto info = detail::get_serialize_runtime_info<conf>(args...);//得到用conf这个配置和需要序列化的args的一些数据,比如下面要用的info.size()是序列化后占用char的大小auto total = data_offset + info.size();//得到总得bufferSizebuffer.resize(total);//重置buffer大小auto writer =struct_pack::detail::memory_writer{(char *)buffer.data() + data_offset};//得到写类 里面有一个成员变量char* buffer和方法write(char* src,size_t len) 把从src长度为len的数据memcpy到buffer处.且buffer会移动到写之后的位置.struct_pack::detail::serialize_to<conf>(writer, info, args...);//把序列化结果写入
}

detail::get_serialize_runtime_info(const Args &…args)

得到运行时的数据

模板类

1. serialize_config conf 配置,见前文描述
2. typename... Args需要序列化的代码.

修饰

1. [[nodiscard]] 返回值不可丢弃
2. #define STRUCT_PACK_INLINE __attribute__((always_inline)) inline 总是内联
3. constexpr

参数

1. 返回serialize_buffer_size主要是存长度(对象序列化后的总长度)和metaInfo_

struct serialize_buffer_size {private:std::size_t len_;unsigned char metainfo_;public:constexpr serialize_buffer_size() : len_(sizeof(uint32_t)), metainfo_(0) {}constexpr std::size_t size() const { return len_; }constexpr unsigned char metainfo() const { return metainfo_; }constexpr operator std::size_t() const { return len_; }template <serialize_config conf, typename... Args>friend STRUCT_PACK_INLINE constexpr serialize_buffer_sizestruct_pack::detail::get_serialize_runtime_info(const Args &...args);
};

2. 需要序列化的对象们const Args &...args.

函数主体

template <serialize_config conf, typename... Args>
[[nodiscard]] STRUCT_PACK_INLINE constexpr serialize_buffer_size
get_serialize_runtime_info(const Args &...args) {using Type = get_args_type<Args...>;//得到需要序列化的对象们的类型constexpr bool has_compatible = serialize_static_config<Type>::has_compatible;//看看类型是否是兼容类型constexpr bool has_type_literal = check_if_add_type_literal<conf, Type>();//看看是否要添加对于类型的描述serialize_buffer_size ret;//返回值auto sz_info = calculate_payload_size(args...);//计算序列化负载的长度//sz_info.max_size存的是最大的单个元素的大小if (sz_info.max_size < (int64_t{1} << 8)) [[likely]] {ret.len_ += sz_info.total + sz_info.size_cnt;ret.metainfo_ = 0b00000;constexpr bool has_compile_time_determined_meta_info =has_compatible || has_type_literal;if constexpr (has_compile_time_determined_meta_info) {ret.len_ += sizeof(unsigned char);}}else {if (sz_info.max_size < (int64_t{1} << 16)) {ret.len_ += sz_info.total + sz_info.size_cnt * 2;ret.metainfo_ = 0b01000;}else if (sz_info.max_size < (int64_t{1} << 32)) {ret.len_ += sz_info.total + sz_info.size_cnt * 4;ret.metainfo_ = 0b10000;}else {ret.len_ += sz_info.total + sz_info.size_cnt * 8;ret.metainfo_ = 0b11000;}// size_type >= 1 , has metainforet.len_ += sizeof(unsigned char);}if constexpr (has_type_literal) {constexpr auto type_literal = struct_pack::get_type_literal<Args...>();// struct_pack::get_type_literal<Args...>().size() crash in clang13. Bug?ret.len_ += type_literal.size() + 1;ret.metainfo_ |= 0b100;}if constexpr (has_compatible) {  // calculate bytes count of serialize// lengthif (ret.len_ + 2 < (int64_t{1} << 16)) [[likely]] {ret.len_ += 2;ret.metainfo_ |= 0b01;}else if (ret.len_ + 4 < (int64_t{1} << 32)) {ret.len_ += 4;ret.metainfo_ |= 0b10;}else {ret.len_ += 8;ret.metainfo_ |= 0b11;}}return ret;
}

get_args_type <Args…>

函数主体

变长参数长度为1时,返回用Args组成的tuple的第0个的类型,即该arg的类型,否则返回用Args组成的tuple的类型

template <typename... Args>
using get_args_type =typename std::conditional<sizeof...(Args) == 1,std::tuple_element_t<0, std::tuple<Args...>>,std::tuple<Args...>>::type;

get_types

得到U的struct_pack需要的类型

函数主体

template <typename U>
constexpr auto get_types() {using T = std::remove_cvref_t<U>;//去类型修饰if constexpr (std::is_fundamental_v<T> || std::is_enum_v<T> || varint_t<T> ||std::is_same_v<std::string, T> || container<T> || optional<T> ||unique_ptr<T> || variant<T> || expected<T> || array<T> ||c_array<T> || std::is_same_v<std::monostate, T>) {return declval<std::tuple<T>>();//如果是基础类型或别的,封装到tuple里,返回一个该类型的右值引用(有引用折叠)}else if constexpr (tuple<T>) {return declval<T>();//本来就是元组,返回一个该类型的右值引用(有引用折叠)}else if constexpr (is_trivial_tuple<T>) {//似乎不会进入这个了,is_trivial_tuple永远为falsereturn declval<T>();}else if constexpr (pair<T>) {//pair也拆进tuple里return declval<std::tuple<typename T::first_type, typename T::second_type>>();}else if constexpr (std::is_aggregate_v<T>) {//是否是聚合类型// clang-format offreturn visit_members(declval<T>(), []<typename... Args>(Args &&...) constexpr {return declval<std::tuple<std::remove_cvref_t<Args>...>>();//如果是,每一个都去修饰});// clang-format on}else {static_assert(!sizeof(T), "the type is not supported!");//否则,说明类型不受支持}
}

check_if_add_type_literal< conf>(args…)

函数主体

根据类型判断要不要加类型描述

template <serialize_config conf, typename T>
constexpr bool check_if_add_type_literal() {if constexpr (conf.add_type_info == type_info_config::automatic) {if constexpr (enable_type_info<T> == type_info_config::automatic) {return serialize_static_config<T>::has_type_literal;}else {return enable_type_info<T> == type_info_config::enable;}}else {return conf.add_type_info == type_info_config::enable;}
}

calculate_payload_size

计算序列化后的大小,dfs计算

函数主体

template <typename T, typename... Args>
constexpr size_info STRUCT_PACK_INLINE
calculate_payload_size(const T &item, const Args &...items) {if constexpr (sizeof...(items))//输入多个参数时,看看第一个和后面的,假如后面的有,则递归求解return calculate_one_size(item) + calculate_payload_size(items...);elsereturn calculate_one_size(item);//只剩一个的时候直接求
}

calculate_one_size

计算一个item的大小

函数主体

template <typename T>
constexpr size_info inline calculate_one_size(const T &item) {constexpr auto id = get_type_id<std::remove_cvref_t<T>>();//得到类型的类型id,几个简单的例子,通用的class和struct都是struct_pack::detail::type_id::trivial_class_tstatic_assert(id != detail::type_id::type_end_flag);//没有对应的id则不合法using type = std::remove_cvref_t<decltype(item)>;//得到item的typestatic_assert(!std::is_pointer_v<type>);//目前不能处理指针size_info ret{.total = 0, .size_cnt = 0, .max_size = 0};//初始化返回值//按类型分类计算if constexpr (id == type_id::monostate_t) {}else if constexpr (std::is_fundamental_v<type> || std::is_enum_v<type>) {ret.total = sizeof(type);//基础类型直接算}else if constexpr (detail::varint_t<type>) {ret.total = detail::calculate_varint_size(item);//varint用函数算}else if constexpr (id == type_id::array_t) {//array看看能不能加,不能则一个一个算if constexpr (is_trivial_serializable<type>::value) {ret.total = sizeof(type);}else {for (auto &i : item) {ret += calculate_one_size(i);//进入类型,逐个算里面的类型}}}else if constexpr (container<type>) {//如果是容器,则按容器的方式计算ret.size_cnt += 1;ret.max_size = (std::max)(ret.max_size, item.size());if constexpr (trivially_copyable_container<type>) {//简单容器直接乘using value_type = typename type::value_type;ret.total = item.size() * sizeof(value_type);}else {for (auto &&i : item) {ret += calculate_one_size(i);//一个一个算}}}else if constexpr (container_adapter<type>) {//如果是容器适配器类型,不支持...确实,主要是,序列化好办,反序列化怎么办static_assert(!sizeof(type), "the container adapter type is not supported");}else if constexpr (!pair<type> && tuple<type> && !is_trivial_tuple<type>) {//如果里面还是个元组,则递归调用算元组的那个函数算std::apply(//std::apply对tuple之类的做元组展开,这里是把item展开填到auto &&...items的位置,已知item是个元组,items里面是元组的元素.[&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {ret += calculate_payload_size(items...);},item);}else if constexpr (optional<type> || unique_ptr<type>) {//如果是optional或者unique_ptr  你看,拥有所有权的时候,就可以做序列化了ret.total = sizeof(char);if (item) {ret += calculate_one_size(*item);}}else if constexpr (variant<type>) {ret.total = sizeof(uint8_t);ret += std::visit(//std::visit用于遍历variant中的所有对象 visit还可以配合lambda的overload{lambda1,lambda2}来进行最优选择[](const auto &e) {return calculate_one_size(e);},item);}else if constexpr (expected<type>) {//用于expectedret.total = sizeof(bool);if (item.has_value()) {if constexpr (!std::is_same_v<typename type::value_type, void>)ret += calculate_one_size(item.value());}else {ret += calculate_one_size(item.error());}}else if constexpr (std::is_class_v<type>) {if constexpr (!pair<type> && !is_trivial_tuple<type>)static_assert(std::is_aggregate_v<std::remove_cvref_t<type>>);if constexpr (is_trivial_serializable<type>::value) {ret.total = sizeof(type);}else if constexpr (is_trivial_serializable<type, true>::value) {visit_members(item, [&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {ret += calculate_payload_size(items...);ret.total += align::total_padding_size<type>;});}else {//对于item里的每一个元素调用后面这个lambda 由于这个visit_member,现在struct_pack的class里最多有64个成员变量visit_members(item, [&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {ret += calculate_payload_size(items...);});}}else {static_assert(!sizeof(type), "the type is not supported yet");}return ret;
}template <trivial_view T>
constexpr size_info inline calculate_one_size(const T &item) {return calculate_one_size(item.get());
}

get_type_id

得到类型的id 建立类型和类型id的映射关系 这里似乎可以优化为统一返回值的形式(如果模板元支持的话…).

函数主体

template <typename T>
consteval type_id get_type_id() {static_assert(CHAR_BIT == 8);// compatible member, which should be ignored in MD5 calculated.if constexpr (optional<T> && is_compatible<T>) {return type_id::compatible_t;}else if constexpr (std::is_enum_v<T>) {return get_integral_type<std::underlying_type_t<T>>();}else if constexpr (std::is_integral_v<T>) {return get_integral_type<T>();}else if constexpr (std::is_floating_point_v<T>) {return get_floating_point_type<T>();}else if constexpr (detail::varint_t<T>) {return get_varint_type<T>();}else if constexpr (std::is_same_v<T, std::monostate> ||std::is_same_v<T, void>) {return type_id::monostate_t;}else if constexpr (string<T>) {return type_id::string_t;}else if constexpr (array<T> || c_array<T> || static_span<T>) {return type_id::array_t;}else if constexpr (map_container<T>) {return type_id::map_container_t;}else if constexpr (set_container<T>) {return type_id::set_container_t;}else if constexpr (container<T>) {return type_id::container_t;}else if constexpr (optional<T> || unique_ptr<T>) {return type_id::optional_t;}else if constexpr (variant<T>) {static_assert(std::variant_size_v<T> > 1 ||(std::variant_size_v<T> == 1 &&!std::is_same_v<std::variant_alternative_t<0, T>, std::monostate>),"The variant should contain's at least one type!");static_assert(std::variant_size_v<T> < 256, "The variant is too complex!");return type_id::variant_t;}else if constexpr (expected<T>) {return type_id::expected_t;}else if constexpr (is_trivial_tuple<T> || pair<T>) {return type_id::trivial_class_t;}else if constexpr (tuple<T>) {return type_id::non_trivial_class_t;}else if constexpr (std::is_class_v<T>) {static_assert(std::is_aggregate_v<std::remove_cvref_t<T>>);return type_id::trivial_class_t;}else {static_assert(!sizeof(T), "not supported type");}
}

serialize_to(Writer &writer, const serialize_buffer_size &info,const Args &…args)

write指明往哪个char buffer里写,info是要写的长度信息,args是要序列化的对象们

模板类

修饰

参数

函数主体

template <serialize_config conf = serialize_config{},struct_pack::writer_t Writer, typename... Args>
STRUCT_PACK_MAY_INLINE void serialize_to(Writer &writer,const serialize_buffer_size &info,const Args &...args) {static_assert(sizeof...(args) > 0);//合法性检查detail::packer<Writer, detail::get_args_type<Args...>> o(writer, info);//实际负责序列化switch ((info.metainfo() & 0b11000) >> 3) {case 0:o.template serialize<conf, 1>(args...);//调用序列化函数break;
#ifdef STRUCT_PACK_OPTIMIZEcase 1:o.template serialize<conf, 2>(args...);break;case 2:o.template serialize<conf, 4>(args...);break;case 3:o.template serialize<conf, 8>(args...);break;
#elsecase 1:case 2:case 3:o.template serialize<conf, 2>(args...);break;
#endifdefault:detail::unreachable();break;};
}

class packer

实际负责序列化的类

成员变量

都是构造时传入的

1. writer &writer_; 往哪写
2. const serialize_buffer_size &info;长度数据

public方法

void serialize(const T &t, const Args &…args)

函数主体

  template <serialize_config conf, std::size_t size_type, typename T,typename... Args>STRUCT_PACK_INLINE void serialize(const T &t, const Args &...args) {serialize_metainfo<conf, size_type == 1, T, Args...>();serialize_many<size_type, UINT64_MAX>(t, args...);using Type = get_args_type<T, Args...>;if constexpr (serialize_static_config<Type>::has_compatible) {constexpr std::size_t sz = compatible_version_number<Type>.size();[&]<std::size_t... I>(std::index_sequence<I...>) {(serialize_many<size_type, compatible_version_number<Type>[I]>(t,args...),...);}(std::make_index_sequence<sz>{});}}

void STRUCT_PACK_INLINE serialize_metainfo()

函数主体

  template <serialize_config conf, bool is_default_size_type, typename T,typename... Args>constexpr void STRUCT_PACK_INLINE serialize_metainfo() {constexpr auto hash_head = calculate_hash_head<conf, T, Args...>() |(is_default_size_type ? 0 : 1);//hash得到一个头writer_.write((char *)&hash_head, sizeof(uint32_t));//写hash的结果if constexpr (hash_head % 2) {  // has more metainfo auto metainfo = info.metainfo();writer_.write((char *)&metainfo, sizeof(char));//写 meta infoif constexpr (serialize_static_config<serialize_type>::has_compatible) {constexpr std::size_t sz[] = {0, 2, 4, 8};auto len_size = sz[metainfo & 0b11];auto len = info.size();writer_.write((char *)&len, len_size);}if constexpr (check_if_add_type_literal<conf, serialize_type>()) {constexpr auto type_literal =struct_pack::get_type_literal<T, Args...>();writer_.write(type_literal.data(), type_literal.size() + 1);}}}

最简单的反序列化

反序列化的内容result被抽象为view,需要满足有data() 和size()两个方法的容器

//expect<对象,error> 
auto person2 = struct_pack::deserialize<person>(result);

auto deserialize(const View &v)

把符合一定条件的类抽象成view类,view类满足一般的连续容器条件.

模板类

1. T要反序列化成的类型
2. typename... Args参数
3. detail::deserialize_view View满足一定条件的view

函数主体

template <typename T, typename... Args, detail::deserialize_view View>//只有一个元素时,Args为空这里可以传入多个类
[[nodiscard]] STRUCT_PACK_INLINE auto deserialize(const View &v) {expected<detail::get_args_type<T, Args...>, struct_pack::errc> ret;//返回值,类或错误的expect,如果传入的是多个类,参见detail::get_args_type<T, Args...>,将得到元组,传入单个参数得到这个类的类型if (auto errc = deserialize_to(ret.value(), v); errc != struct_pack::errc{})//用更基础的函数deserialize_to[[unlikely]] {ret = unexpected<struct_pack::errc>{errc};//非空则有异常}return ret;
}

template <typename T, typename… Args, detail::deserialize_view View>[[nodiscard]] STRUCT_PACK_INLINE struct_pack::errc deserialize_to( T &t, const View &v, Args &…args)

函数主体

template <typename T, typename... Args, detail::deserialize_view View>
[[nodiscard]] STRUCT_PACK_INLINE struct_pack::errc deserialize_to(T &t, const View &v, Args &...args) {//T&t写对象的位置 View &v反序列化的容器detail::memory_reader reader{(const char *)v.data(),(const char *)v.data() + v.size()};//构造读取器 限定读写内存边界detail::unpacker in(reader);//用读取器初始化反序列化类return in.deserialize(t, args...);//反序列化
}

class unpacker

用于反序列化的类

public方法

template <typename T, typename… Args> STRUCT_PACK_MAY_INLINE struct_pack::errc deserialize(T &t, Args &…args)

代码

  template <typename T, typename... Args>STRUCT_PACK_MAY_INLINE struct_pack::errc deserialize(T &t, Args &...args) {using Type = get_args_type<T, Args...>;//单个的得到类型,多个的得到tupleconstexpr bool has_compatible =check_if_compatible_element_exist<decltype(get_types<Type>())>();//是否满足兼容性if constexpr (has_compatible) {data_len_ = reader_.tellg();}auto &&[err_code, buffer_len] = deserialize_metainfo<Type>();if (err_code != struct_pack::errc{}) [[unlikely]] {return err_code;}if constexpr (has_compatible) {data_len_ += buffer_len;}switch (size_type_) {case 0:err_code = deserialize_many<1, UINT64_MAX>(t, args...);break;
#ifdef STRUCT_PACK_OPTIMIZEcase 1:err_code = deserialize_many<2, UINT64_MAX>(t, args...);break;case 2:err_code = deserialize_many<4, UINT64_MAX>(t, args...);break;case 3:err_code = deserialize_many<8, UINT64_MAX>(t, args...);break;
#elsecase 1:case 2:case 3:err_code = deserialize_many<2, UINT64_MAX>(t, args...);break;
#endifdefault:unreachable();}if constexpr (has_compatible) {if (err_code != errc::ok) [[unlikely]] {return err_code;}constexpr std::size_t sz = compatible_version_number<Type>.size();err_code = deserialize_compatibles<T, Args...>(t, args..., std::make_index_sequence<sz>{});}return err_code;}

template <size_t size_type, uint64_t version, bool NotSkip = true> constexpr struct_pack::errc STRUCT_PACK_INLINE deserialize_many(auto &&first_item, auto &&…items)

函数主体

  template <size_t size_type, uint64_t version, bool NotSkip = true>constexpr struct_pack::errc STRUCT_PACK_INLINEdeserialize_many(auto &&first_item, auto &&...items) {auto code = deserialize_one<size_type, version, NotSkip>(first_item);if (code != struct_pack::errc{}) [[unlikely]] {return code;}return deserialize_many<size_type, version, NotSkip>(items...);}

template <size_t size_type, uint64_t version, bool NotSkip> constexpr struct_pack::errc inline deserialize_one(auto &item)

模板类

修饰

参数

函数主体

  template <size_t size_type, uint64_t version, bool NotSkip>constexpr struct_pack::errc inline deserialize_one(auto &item) {struct_pack::errc code{};//错误码using type = std::remove_cvref_t<decltype(item)>;//拿去修饰的类型static_assert(!std::is_pointer_v<type>);//不能是指针constexpr auto id = get_type_id<type>();//由类型得到类型的idif constexpr (version == UINT64_MAX) {//判断版本,目前似乎是写死的if constexpr (id == type_id::compatible_t) {//逐个判断类型// do nothing}else if constexpr (std::is_same_v<type, std::monostate>) {// do nothing}else if constexpr (std::is_fundamental_v<type> || std::is_enum_v<type>) {//基础类型,读一个大小直接塞到item里if constexpr (NotSkip) {if (!reader_.read((char *)&item, sizeof(type))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}}else {return reader_.ignore(sizeof(type)) ? errc{} : errc::no_buffer_space;}}else if constexpr (detail::varint_t<type>) {code = detail::deserialize_varint<NotSkip>(reader_, item);}else if constexpr (id == type_id::array_t) {if constexpr (is_trivial_serializable<type>::value) {if constexpr (NotSkip) {if (!reader_.read((char *)&item, sizeof(type))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}}else {return reader_.ignore(sizeof(type)) ? errc{}: errc::no_buffer_space;}}else {for (auto &i : item) {code = deserialize_one<size_type, version, NotSkip>(i);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}}else if constexpr (container<type>) {size_t size = 0;
#ifdef STRUCT_PACK_OPTIMIZEif (!reader_.read((char *)&size, size_type)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}
#elseif constexpr (size_type == 1) {if (!reader_.read((char *)&size, size_type)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}}else {switch (size_type_) {case 1:if (!reader_.read((char *)&size, 2)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}break;case 2:if (!reader_.read((char *)&size, 4)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}break;case 3:if (!reader_.read((char *)&size, 8)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}break;default:unreachable();}}
#endifif (size == 0) [[unlikely]] {return {};}if constexpr (map_container<type> || set_container<type>) {// value is the element of map/set container.// if the type is set, then value is set::value_type;// if the type is map, then value is pair<key_type,mapped_type>decltype([]<typename T>(const T &) {if constexpr (map_container<T>) {return std::pair<typename T::key_type, typename T::mapped_type>{};}else {return typename T::value_type{};}}(item)) value;if constexpr (is_trivial_serializable<decltype(value)>::value &&!NotSkip) {return reader_.ignore(size * sizeof(value)) ? errc{}: errc::no_buffer_space;}else {for (size_t i = 0; i < size; ++i) {code = deserialize_one<size_type, version, NotSkip>(value);if (code != struct_pack::errc{}) [[unlikely]] {return code;}if constexpr (NotSkip) {item.emplace(std::move(value));// TODO: mapped_type can deserialize without be moved}}}}else {using value_type = typename type::value_type;if constexpr (trivially_copyable_container<type>) {size_t mem_sz = size * sizeof(value_type);if constexpr (NotSkip) {if constexpr (string_view<type> || dynamic_span<type>) {static_assert(view_reader_t<Reader>,"The Reader isn't a view_reader, can't deserialize ""a string_view/span");const char *view = reader_.read_view(mem_sz);if (view == nullptr) [[unlikely]] {return struct_pack::errc::no_buffer_space;}item = {(value_type *)(view), size};}else {if (mem_sz >= PTRDIFF_MAX) [[unlikely]]unreachable();else {item.resize(size);if (!reader_.read((char *)item.data(), mem_sz)) [[unlikely]] {return struct_pack::errc::no_buffer_space;}}}}else {return reader_.ignore(mem_sz) ? errc{} : errc::no_buffer_space;}}else {if constexpr (NotSkip) {if constexpr (dynamic_span<type>) {static_assert(!dynamic_span<type>,"It's illegal to deserialize a span<T> which T ""is a non-trival-serializable type.");}else {item.resize(size);for (auto &i : item) {code = deserialize_one<size_type, version, NotSkip>(i);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}}else {value_type useless;for (size_t i = 0; i < size; ++i) {code = deserialize_one<size_type, version, NotSkip>(useless);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}}}}else if constexpr (container_adapter<type>) {static_assert(!sizeof(type),"the container adapter type is not supported");}else if constexpr (!pair<type> && tuple<type> &&!is_trivial_tuple<type>) {std::apply([&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {code = deserialize_many<size_type, version, NotSkip>(items...);},item);}else if constexpr (optional<type> || expected<type>) {bool has_value;if (!reader_.read((char *)&has_value, sizeof(bool))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}if (!has_value) [[unlikely]] {if constexpr (expected<type>) {item = typename type::unexpected_type{typename type::error_type{}};deserialize_one<size_type, version, NotSkip>(item.error());}else {return {};}}else {if constexpr (expected<type>) {if constexpr (!std::is_same_v<typename type::value_type, void>)deserialize_one<size_type, version, NotSkip>(item.value());}else {item = type{std::in_place_t{}};deserialize_one<size_type, version, NotSkip>(*item);}}}else if constexpr (variant<type>) {uint8_t index;if (!reader_.read((char *)&index, sizeof(index))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}if (index >= std::variant_size_v<type>) [[unlikely]] {return struct_pack::errc::invalid_buffer;}else {template_switch<variant_construct_helper,std::integral_constant<std::size_t, size_type>,std::integral_constant<std::uint64_t, version>,std::integral_constant<bool, NotSkip>>(index, *this,item);}}else if constexpr (std::is_class_v<type>) {//如果是类if constexpr (!pair<type> && !is_trivial_tuple<type>)//如果是元组static_assert(std::is_aggregate_v<std::remove_cvref_t<type>>);if constexpr (is_trivial_serializable<type>::value) {//简单线性if constexpr (NotSkip) {if (!reader_.read((char *)&item, sizeof(type))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}}else {return reader_.ignore(sizeof(type)) ? errc{}: errc::no_buffer_space;}}else if constexpr (is_trivial_serializable<type, true>::value) {visit_members(item, [&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {int i = 1;[[maybe_unused]] bool op =(([&]() {if (code = deserialize_one<size_type, version, NotSkip>(items);code == errc::ok) [[likely]] {code = ignore_padding(align::padding_size<type>[i]);++i;}}(),code == errc::ok) &&...);});}else {//逐个访问元素 假定每个子元素还是多个的进行访问,单个的最终会被deserialize_one接收.visit_members(item, [&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {code = deserialize_many<size_type, version, NotSkip>(items...);});}}else {static_assert(!sizeof(type), "the type is not supported yet");}}else if constexpr (exist_compatible_member<type, version>) {if constexpr (id == type_id::compatible_t) {if constexpr (version == type::version_number) {auto pos = reader_.tellg();if ((std::size_t)pos >= data_len_) {if (std::is_unsigned_v<decltype(pos)> || pos >= 0) {size_type_ = UCHAR_MAX;  // Just notice that this is not a real// error, this is a flag for exit.}return struct_pack::errc::no_buffer_space;}bool has_value;if (!reader_.read((char *)&has_value, sizeof(bool))) [[unlikely]] {return struct_pack::errc::no_buffer_space;}if (!has_value) {return code;}else {item = type{std::in_place_t{}};deserialize_one<size_type, UINT64_MAX, NotSkip>(*item);}}}else if constexpr (id == type_id::array_t) {for (auto &i : item) {code = deserialize_one<size_type, version, NotSkip>(i);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}else if constexpr (container<type>) {if constexpr (id == type_id::set_container_t) {// TODO: support it.static_assert(!sizeof(type),"we don't support compatible field in set now.");}else if constexpr (id == type_id::map_container_t) {static_assert(!exist_compatible_member<typename type::key_type>,"we don't support compatible field in map's key_type now.");if constexpr (NotSkip) {for (auto &e : item) {code = deserialize_one<size_type, version, NotSkip>(e.second);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}else {// TODO: support it.static_assert(!sizeof(type),"we don't support skip compatible field in container now.");}// how to deserialize it quickly?}else {if constexpr (NotSkip) {for (auto &i : item) {code = deserialize_one<size_type, version, NotSkip>(i);if (code != struct_pack::errc{}) [[unlikely]] {return code;}}}else {// TODO: support it.static_assert(!sizeof(type),"we don't support skip compatible field in container now.");}}}else if constexpr (!pair<type> && tuple<type> &&!is_trivial_tuple<type>) {std::apply([&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {code = deserialize_many<size_type, version, NotSkip>(items...);},item);}else if constexpr (optional<type> || expected<type>) {bool has_value = item.has_value();if (!has_value) {if constexpr (expected<type>) {deserialize_one<size_type, version, NotSkip>(item.error());}}else {if constexpr (expected<type>) {if constexpr (!std::is_same_v<typename type::value_type, void>)deserialize_one<size_type, version, NotSkip>(item.value());}else {deserialize_one<size_type, version, NotSkip>(item.value());}}}else if constexpr (variant<type>) {std::visit([this](auto &item) {deserialize_one<size_type, version, NotSkip>(item);},item);}else if constexpr (std::is_class_v<type>) {visit_members(item, [&](auto &&...items) CONSTEXPR_INLINE_LAMBDA {code = deserialize_many<size_type, version, NotSkip>(items...);});}}return code;}