// Package idxfile implements encoding and decoding of packfile idx files. // // == Original (version 1) pack-*.idx files have the following format: // // - The header consists of 256 4-byte network byte order // integers. N-th entry of this table records the number of // objects in the corresponding pack, the first byte of whose // object name is less than or equal to N. This is called the // 'first-level fan-out' table. // // - The header is followed by sorted 24-byte entries, one entry // per object in the pack. Each entry is: // // 4-byte network byte order integer, recording where the // object is stored in the packfile as the offset from the // beginning. // // 20-byte object name. // // - The file is concluded with a trailer: // // A copy of the 20-byte SHA1 checksum at the end of // corresponding packfile. // // 20-byte SHA1-checksum of all of the above. // // Pack Idx file: // // -- +--------------------------------+ // fanout | fanout[0] = 2 (for example) |-. // table +--------------------------------+ | // | fanout[1] | | // +--------------------------------+ | // | fanout[2] | | // ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | // | fanout[255] = total objects |---. // -- +--------------------------------+ | | // main | offset | | | // index | object name 00XXXXXXXXXXXXXXXX | | | // tab +--------------------------------+ | | // | offset | | | // | object name 00XXXXXXXXXXXXXXXX | | | // +--------------------------------+<+ | // .-| offset | | // | | object name 01XXXXXXXXXXXXXXXX | | // | +--------------------------------+ | // | | offset | | // | | object name 01XXXXXXXXXXXXXXXX | | // | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | // | | offset | | // | | object name FFXXXXXXXXXXXXXXXX | | // --| +--------------------------------+<--+ // trailer | | packfile checksum | // | +--------------------------------+ // | | idxfile checksum | // | +--------------------------------+ // .---------. // | // Pack file entry: <+ // // packed object header: // 1-byte size extension bit (MSB) // type (next 3 bit) // size0 (lower 4-bit) // n-byte sizeN (as long as MSB is set, each 7-bit) // size0..sizeN form 4+7+7+..+7 bit integer, size0 // is the least significant part, and sizeN is the // most significant part. // packed object data: // If it is not DELTA, then deflated bytes (the size above // is the size before compression). // If it is REF_DELTA, then // 20-byte base object name SHA1 (the size above is the // size of the delta data that follows). // delta data, deflated. // If it is OFS_DELTA, then // n-byte offset (see below) interpreted as a negative // offset from the type-byte of the header of the // ofs-delta entry (the size above is the size of // the delta data that follows). // delta data, deflated. // // offset encoding: // n bytes with MSB set in all but the last one. // The offset is then the number constructed by // concatenating the lower 7 bit of each byte, and // for n >= 2 adding 2^7 + 2^14 + ... + 2^(7*(n-1)) // to the result. // // == Version 2 pack-*.idx files support packs larger than 4 GiB, and // have some other reorganizations. They have the format: // // - A 4-byte magic number '\377tOc' which is an unreasonable // fanout[0] value. // // - A 4-byte version number (= 2) // // - A 256-entry fan-out table just like v1. // // - A table of sorted 20-byte SHA1 object names. These are // packed together without offset values to reduce the cache // footprint of the binary search for a specific object name. // // - A table of 4-byte CRC32 values of the packed object data. // This is new in v2 so compressed data can be copied directly // from pack to pack during repacking without undetected // data corruption. // // - A table of 4-byte offset values (in network byte order). // These are usually 31-bit pack file offsets, but large // offsets are encoded as an index into the next table with // the msbit set. // // - A table of 8-byte offset entries (empty for pack files less // than 2 GiB). Pack files are organized with heavily used // objects toward the front, so most object references should // not need to refer to this table. // // - The same trailer as a v1 pack file: // // A copy of the 20-byte SHA1 checksum at the end of // corresponding packfile. // // 20-byte SHA1-checksum of all of the above. // // Source: // https://www.kernel.org/pub/software/scm/git/docs/v1.7.5/technical/pack-format.txt package idxfile