张家港阿里网站建设/2023百度秒收录技术

17.4 Java堆空间内存分配

分配Java堆内存前，我们先通过两图来了解下C堆、Java堆、内核空间、native本地空间的关系。

1、从图17-1来看，Java堆的分配其实就是从Java进程运行时堆中选中一块内存区域来映射

2、从图17-2，可以看中各内存空间的关系，当然实际的内存区域比这个复杂的多，这里只是概括说明

图17-1

图17-2

17.4.1 genCollectedHeap.cpp

17.4.1.1 GenCollectedHeap::initialize

jint GenCollectedHeap::initialize() {// 这一步只是对c2编译器开通使用时，做一些参数赋值操作，这里就不展开讲CollectedHeap::pre_initialize();// 这里获取分代数_n_gens，就是2int i;_n_gens = gen_policy()->number_of_generations();// 保证2个值相等wordSize和HeapWordSize分别是在操作系统和Java堆中代表一个字word占用内存的大小，这两个值必然相同，否则出错guarantee(HeapWordSize == wordSize, "HeapWordSize must equal wordSize");// Java堆的对齐值，这个在`章节17.2.1.1`中有介绍size_t gen_alignment = Generation::GenGrain;// 获取分代对象数组，这个在`章节17.2.1.1`中有介绍，数组元素就2个，索引0元素表示年轻代，索引1元素表示老年代_gen_specs = gen_policy()->generations();// 分别遍历新生代和老年代，并设置各自分代的空间大小（初始值和最大值），同时确保内存对齐for (i = 0; i < _n_gens; i++) {_gen_specs[i]->align(gen_alignment);}// 下面才是给Java堆分配空间char* heap_address;size_t total_reserved = 0;int n_covered_regions = 0;ReservedSpace heap_rs;// 这是最外层Java堆的内存对齐值size_t heap_alignment = collector_policy()->heap_alignment();// 分配java堆内存，看`章节17.4.1.2`heap_address = allocate(heap_alignment, &total_reserved,&n_covered_regions, &heap_rs);if (!heap_rs.is_reserved()) {vm_shutdown_during_initialization("Could not reserve enough space for object heap");return JNI_ENOMEM;}// 将分配的Java堆内存，用 MemRegion 内存区域对象管理起来_reserved = MemRegion((HeapWord*)heap_rs.base(),(HeapWord*)(heap_rs.base() + heap_rs.size()));// 参数赋值_reserved.set_word_size(0);_reserved.set_start((HeapWord*)heap_rs.base()); // Java堆内存的首地址size_t actual_heap_size = heap_rs.size(); // Java堆内存大小// Java堆内存的限制地址，也就是不能超过这条线_reserved.set_end((HeapWord*)(heap_rs.base() + actual_heap_size)); // 接下来就是创建记忆集、卡表的过程，卡表和记忆集都是为了解决跨代引用的实现方案，后续讲GC时会有涉及_rem_set = collector_policy()->create_rem_set(_reserved, n_covered_regions);set_barrier_set(rem_set()->bs());_gch = this;for (i = 0; i < _n_gens; i++) {ReservedSpace this_rs = heap_rs.first_part(_gen_specs[i]->max_size(), false, false);_gens[i] = _gen_specs[i]->init(this_rs, i, rem_set());heap_rs = heap_rs.last_part(_gen_specs[i]->max_size());}clear_incremental_collection_failed();#if INCLUDE_ALL_GCS// If we are running CMS, create the collector responsible// for collecting the CMS generations.if (collector_policy()->is_concurrent_mark_sweep_policy()) {bool success = create_cms_collector();if (!success) return JNI_ENOMEM;}
#endif // INCLUDE_ALL_GCSreturn JNI_OK;
}

17.4.1.2 GenCollectedHeap::allocate

char* GenCollectedHeap::allocate(size_t alignment,size_t* _total_reserved,int* _n_covered_regions,ReservedSpace* heap_rs){const char overflow_msg[] = "The size of the object heap + VM data exceeds ""the maximum representable size";// Now figure out the total size.size_t total_reserved = 0;int n_covered_regions = 0;const size_t pageSize = UseLargePages ?os::large_page_size() : os::vm_page_size();assert(alignment % pageSize == 0, "Must be");// 遍历_gen_specs，求得新生代和老年代的分配大小for (int i = 0; i < _n_gens; i++) {total_reserved += _gen_specs[i]->max_size();if (total_reserved < _gen_specs[i]->max_size()) {vm_exit_during_initialization(overflow_msg);}n_covered_regions += _gen_specs[i]->n_covered_regions();  // 最终为2}assert(total_reserved % alignment == 0,err_msg("Gen size; total_reserved=" SIZE_FORMAT ", alignment="SIZE_FORMAT, total_reserved, alignment));// Needed until the cardtable is fixed to have the right number// of covered regions.n_covered_regions += 2;  // 再加2，就是4，也就是把堆最终分成4个区（新生代、S1、S2、老年代）*_total_reserved = total_reserved;*_n_covered_regions = n_covered_regions;// 分配内存，实现细节看`章节17.4.2`*heap_rs = Universe::reserve_heap(total_reserved, alignment);return heap_rs->base();
}

17.4.2 universe.cpp

17.4.2.1 Universe::reserve_heap

ReservedSpace Universe::reserve_heap(size_t heap_size, size_t alignment) {assert(alignment <= Arguments::conservative_max_heap_alignment(),err_msg("actual alignment " SIZE_FORMAT " must be within maximum heap alignment " SIZE_FORMAT,alignment, Arguments::conservative_max_heap_alignment()));// 通过内存对齐，得到要分配的空间大小size_t total_reserved = align_size_up(heap_size, alignment);assert(!UseCompressedOops || (total_reserved <= (OopEncodingHeapMax - os::vm_page_size())),"heap size is too big for compressed oops");// 大页时考虑，本系列文章中不考虑大而情况，忽略bool use_large_pages = UseLargePages && is_size_aligned(alignment, os::large_page_size());assert(!UseLargePages|| UseParallelGC|| use_large_pages, "Wrong alignment to use large pages");// 取出Java堆的基址base的值，32位机器时，就是0，实现细节看`章节17.4.2.2`char* addr = Universe::preferred_heap_base(total_reserved, alignment, Universe::UnscaledNarrowOop);// 创建一个ReservedHeapSpace对象，该对象就是用来保留连续内存地址范围空间的数据结构，实现细节看`章节17.4.3`ReservedHeapSpace total_rs(total_reserved, alignment, use_large_pages, addr);if (UseCompressedOops) {if (addr != NULL && !total_rs.is_reserved()) {// Failed to reserve at specified address - the requested memory// region is taken already, for example, by 'java' launcher.// Try again to reserver heap higher.addr = Universe::preferred_heap_base(total_reserved, alignment, Universe::ZeroBasedNarrowOop);ReservedHeapSpace total_rs0(total_reserved, alignment,use_large_pages, addr);if (addr != NULL && !total_rs0.is_reserved()) {// Failed to reserve at specified address again - give up.addr = Universe::preferred_heap_base(total_reserved, alignment, Universe::HeapBasedNarrowOop);assert(addr == NULL, "");ReservedHeapSpace total_rs1(total_reserved, alignment,use_large_pages, addr);total_rs = total_rs1;} else {total_rs = total_rs0;}}}if (!total_rs.is_reserved()) {vm_exit_during_initialization(err_msg("Could not reserve enough space for " SIZE_FORMAT "KB object heap", total_reserved/K));return total_rs;}if (UseCompressedOops) {// Universe::initialize_heap() will reset this to NULL if unscaled// or zero-based narrow oops are actually used.address base = (address)(total_rs.base() - os::vm_page_size());Universe::set_narrow_oop_base(base);}// 返回total_rsreturn total_rs;
}

17.4.2.2 Universe::preferred_heap_base

char* Universe::preferred_heap_base(size_t heap_size, size_t alignment, NARROW_OOP_MODE mode) {assert(is_size_aligned((size_t)OopEncodingHeapMax, alignment), "Must be");assert(is_size_aligned((size_t)UnscaledOopHeapMax, alignment), "Must be");assert(is_size_aligned(heap_size, alignment), "Must be");// HeapBaseMinAddress 是操作系统明确设定的堆内存的最低地址限制，默认设置的是2*G，这里按alignment对齐，把HeapBaseMinAddress的值按alignment对齐后，作为堆内存的最低地址uintx heap_base_min_address_aligned = align_size_up(HeapBaseMinAddress, alignment);size_t base = 0;
#ifdef _LP64  // 下面是对64位机器及使用压缩指针时的实现，我们只讲32位的，这块逻辑略过if (UseCompressedOops) {assert(mode == UnscaledNarrowOop  ||mode == ZeroBasedNarrowOop ||mode == HeapBasedNarrowOop, "mode is invalid");const size_t total_size = heap_size + heap_base_min_address_aligned;// Return specified base for the first request.if (!FLAG_IS_DEFAULT(HeapBaseMinAddress) && (mode == UnscaledNarrowOop)) {base = heap_base_min_address_aligned;// If the total size is small enough to allow UnscaledNarrowOop then// just use UnscaledNarrowOop.} else if ((total_size <= OopEncodingHeapMax) && (mode != HeapBasedNarrowOop)) {if ((total_size <= UnscaledOopHeapMax) && (mode == UnscaledNarrowOop) &&(Universe::narrow_oop_shift() == 0)) {// Use 32-bits oops without encoding and// place heap's top on the 4Gb boundarybase = (UnscaledOopHeapMax - heap_size);} else {// Can't reserve with NarrowOopShift == 0Universe::set_narrow_oop_shift(LogMinObjAlignmentInBytes);if (mode == UnscaledNarrowOop ||mode == ZeroBasedNarrowOop && total_size <= UnscaledOopHeapMax) {// Use zero based compressed oops with encoding and// place heap's top on the 32Gb boundary in case// total_size > 4Gb or failed to reserve below 4Gb.uint64_t heap_top = OopEncodingHeapMax;// For small heaps, save some space for compressed class pointer// space so it can be decoded with no base.if (UseCompressedClassPointers && !UseSharedSpaces &&OopEncodingHeapMax <= 32*G) {uint64_t class_space = align_size_up(CompressedClassSpaceSize, alignment);assert(is_size_aligned((size_t)OopEncodingHeapMax-class_space,alignment), "difference must be aligned too");uint64_t new_top = OopEncodingHeapMax-class_space;if (total_size <= new_top) {heap_top = new_top;}}// Align base to the adjusted top of the heapbase = heap_top - heap_size;}}} else {// UnscaledNarrowOop encoding didn't work, and no base was found for ZeroBasedOops or// HeapBasedNarrowOop encoding was requested.  So, can't reserve below 32Gb.Universe::set_narrow_oop_shift(LogMinObjAlignmentInBytes);}// Set narrow_oop_base and narrow_oop_use_implicit_null_checks// used in ReservedHeapSpace() constructors.// The final values will be set in initialize_heap() below.if ((base != 0) && ((base + heap_size) <= OopEncodingHeapMax)) {// Use zero based compressed oopsUniverse::set_narrow_oop_base(NULL);// Don't need guard page for implicit checks in indexed// addressing mode with zero based Compressed Oops.Universe::set_narrow_oop_use_implicit_null_checks(true);} else {// Set to a non-NULL value so the ReservedSpace ctor computes// the correct no-access prefix.// The final value will be set in initialize_heap() below.Universe::set_narrow_oop_base((address)UnscaledOopHeapMax);
#if defined(_WIN64) || defined(AIX)if (UseLargePages) {// Cannot allocate guard pages for implicit checks in indexed// addressing mode when large pages are specified on windows.Universe::set_narrow_oop_use_implicit_null_checks(false);}
#endif //  _WIN64}}
#endifassert(is_ptr_aligned((char*)base, alignment), "Must be");// 最终返回base,在32位机器时，虚拟机就是返回0return (char*)base; // also return NULL (don't care) for 32-bit VM
}

17.4.3 virtualspace.cpp

17.4.3.1 ReservedHeapSpace::ReservedHeapSpace

ReservedHeapSpace::ReservedHeapSpace(size_t size, size_t alignment,bool large, char* requested_address) :/* 先调用父类构造函数*/ReservedSpace(size, alignment, large,requested_address,(UseCompressedOops && (Universe::narrow_oop_base() != NULL) &&Universe::narrow_oop_use_implicit_null_checks()) ?lcm(os::vm_page_size(), alignment) : 0) {if (base() != NULL) {MemTracker::record_virtual_memory_type((address)base(), mtJavaHeap);}// Only reserved space for the java heap should have a noaccess_prefix// if using compressed oops.protect_noaccess_prefix(size);
}

17.4.3.2 ReservedSpace::ReservedSpace

ReservedSpace::ReservedSpace(size_t size, size_t alignment,bool large,char* requested_address,const size_t noaccess_prefix) {initialize(size+noaccess_prefix, alignment, large, requested_address,noaccess_prefix, false);
}

17.4.3.3 ReservedSpace::initialize

入口函数： ReservedHeapSpace total_rs(total_reserved, alignment, use_large_pages, addr);

参数：

total_reserved 对应 size：空间大小

alignment 对应 alignment：内存对齐值

use_large_pages 对应 large：这里不考虑大页，就设置为false

addr 对应 requested_address：32位时，addr为0

noaccess_prefix 为 0

executable 为 false

void ReservedSpace::initialize(size_t size, size_t alignment, bool large,char* requested_address,const size_t noaccess_prefix,bool executable) {// 看源码得知，这里就是取page size（页大小），没什么逻辑const size_t granularity = os::vm_allocation_granularity();// 断言检验assert((size & (granularity - 1)) == 0,"size not aligned to os::vm_allocation_granularity()");assert((alignment & (granularity - 1)) == 0,"alignment not aligned to os::vm_allocation_granularity()");assert(alignment == 0 || is_power_of_2((intptr_t)alignment),"not a power of 2");// 取二者最大值对齐alignment = MAX2(alignment, (size_t)os::vm_page_size());// Assert that if noaccess_prefix is used, it is the same as alignment.assert(noaccess_prefix == 0 ||noaccess_prefix == alignment, "noaccess prefix wrong");_base = NULL;_size = 0;_special = false;_executable = executable;_alignment = 0;_noaccess_prefix = 0;if (size == 0) {return;}// 不存在大页，special 为 falsebool special = large && !os::can_commit_large_page_memory();char* base = NULL;// 32位机器时 requested_address == 0，这条线也不会走if (requested_address != 0) {requested_address -= noaccess_prefix; // adjust requested addressassert(requested_address != NULL, "huge noaccess prefix?");}// special为false,这个if不会走if (special) {base = os::reserve_memory_special(size, alignment, requested_address, executable);if (base != NULL) {if (failed_to_reserve_as_requested(base, requested_address, size, true)) {// OS ignored requested address. Try different address.return;}// Check alignment constraints.assert((uintptr_t) base % alignment == 0,err_msg("Large pages returned a non-aligned address, base: "PTR_FORMAT " alignment: " PTR_FORMAT,base, (void*)(uintptr_t)alignment));_special = true;} else {// failed; try to reserve regular memory belowif (UseLargePages && (!FLAG_IS_DEFAULT(UseLargePages) ||!FLAG_IS_DEFAULT(LargePageSizeInBytes))) {if (PrintCompressedOopsMode) {tty->cr();tty->print_cr("Reserve regular memory without large pages.");}}}}if (base == NULL) {if (requested_address != 0) {base = os::attempt_reserve_memory_at(size, requested_address);if (failed_to_reserve_as_requested(base, requested_address, size, false)) {// OS ignored requested address. Try different address.base = NULL;}} else {// 这一步就是通过系统调用mmap映射一块size大小的内存，Java堆内存就是mmap映射出来的base = os::reserve_memory(size, NULL, alignment);}// 映射失败，直接退出函数，分配Java堆内存失败if (base == NULL) return;// 验证对齐，为啥要验证呢，因为base是mmap映射后返回的内存首地址，这个地址是os自己的规则选取的一个地址，不一定能按照alignment对齐，所以这一定要验证if ((((size_t)base + noaccess_prefix) & (alignment - 1)) != 0) {// base没有对齐，只能释放刚才mmap映射的内存，然后重试if (!os::release_memory(base, size)) fatal("os::release_memory failed");// 确保对齐size = align_size_up(size, alignment);// 再次mmap映射内存，返回的base同样有上面一样的不对齐问题，所以这个函数中包含了手动对齐操作，细节看`章节17.4.3.4`base = os::reserve_memory_aligned(size, alignment);if (requested_address != 0 &&failed_to_reserve_as_requested(base, requested_address, size, false)) {// As a result of the alignment constraints, the allocated base differs// from the requested address. Return back to the caller who can// take remedial action (like try again without a requested address).assert(_base == NULL, "should be");return;}}}// Done_base = base;  // 最终拿到了Java堆的首地址_size = size;  // 最终拿到了Java堆的大小_alignment = alignment;  // 对齐值_noaccess_prefix = noaccess_prefix;  // 0// 断言判断assert(noaccess_prefix == 0 ||noaccess_prefix == _alignment, "noaccess prefix wrong");assert(markOopDesc::encode_pointer_as_mark(_base)->decode_pointer() == _base,"area must be distinguisable from marks for mark-sweep");assert(markOopDesc::encode_pointer_as_mark(&_base[size])->decode_pointer() == &_base[size],"area must be distinguisable from marks for mark-sweep");
}

17.4.3.4 os_posix.cpp->os::reserve_memory_aligned

char* os::reserve_memory_aligned(size_t size, size_t alignment) {assert((alignment & (os::vm_allocation_granularity() - 1)) == 0,"Alignment must be a multiple of allocation granularity (page size)");assert((size & (alignment -1)) == 0, "size must be 'alignment' aligned");size_t extra_size = size + alignment;assert(extra_size >= size, "overflow, size is too large to allow alignment");// mmap映射一块内存区域，返回首地址char* extra_base = os::reserve_memory(extra_size, NULL, alignment);if (extra_base == NULL) {return NULL;}// 手动对齐char* aligned_base = (char*) align_size_up((uintptr_t) extra_base, alignment);// [  |                                       |  ]// ^ extra_base//    ^ extra_base + begin_offset == aligned_base//     extra_base + begin_offset + size       ^//                       extra_base + extra_size ^// |<>| == begin_offset//                              end_offset == |<>|// 用对齐后的地址-mmap的首地址，得出与首地址的偏移值size_t begin_offset = aligned_base - extra_base;// 结束地址对齐后的偏移size_t end_offset = (extra_base + extra_size) - (aligned_base + size);// begin_offset > 0，表示确实有偏移，那就把extra_base到偏移的这部分释放掉，因为有新的首地址了if (begin_offset > 0) {os::release_memory(extra_base, begin_offset);}// end_offset > 0，表示确实有偏移，那就把end_offset偏移的这部分释放掉，因为有新的限制地址了if (end_offset > 0) {os::release_memory(extra_base + begin_offset + size, end_offset);}// 返回首地址return aligned_base;
}