aptly/files
Ryan Gonzalez 8ab8398c50 Use github.com/saracen/walker for file walk operations
In some local tests w/ a slowed down filesystem, this massively cut down
on the time to clean up a repository by ~3x, bringing a total 'publish
update' time from ~16s to ~13s.

Signed-off-by: Ryan Gonzalez <ryan.gonzalez@collabora.com>
2024-04-24 16:46:16 +02:00
..
README.md Add -json output flag to repo list|show 2021-09-24 10:29:33 +02:00
files.go Refactor Repository: split into PackagePool and PublishedStorage. 2014-02-19 12:03:01 +04:00
files_test.go Upgrade gocheck 2014-11-05 13:27:15 -06:00
mocks.go Fix paths after repository transfer to aptly-dev 2018-04-18 21:19:43 +03:00
package_pool.go Use github.com/saracen/walker for file walk operations 2024-04-24 16:46:16 +02:00
package_pool_test.go Fix pure-go unittests 2022-01-27 09:30:14 +01:00
public.go Use github.com/saracen/walker for file walk operations 2024-04-24 16:46:16 +02:00
public_test.go Cache bucket content by prefix 2024-02-06 20:49:35 +01:00

README.md

Downloaded packages

SHA-256

For each uploaded Debian package a SHA-256 checksum is computed. This checksum is used to create a file tree where each package will reside, with following hierarchy.

Directory and sub-directories structure

  • 1st and 2nd characters of SHA-256 checksum as sub-directory of rootDir/pool directory.
  • 3rd and 4th characters of SHA-256 checksum as sub-directory of the former

ex:

sha256sum 476e0cdac6bc757dd2b78bacc1325323b09c45ecb41d4562deec2a1c7c148405 my-package_1.2.3_all.deb

${rootDir}/pool # rootDir defined in aptly.conf
└── 47
    └── 6e

Filename

The following items are concatenated to form the filename under which package is stored.

  • 5th to the 31st characters of SHA-256 checksum
  • "_" (undescore)
  • filename of uploaded Debian as defined in Debian package file names

ex:

sha256sum 476e0cdac6bc757dd2b78bacc1325323b09c45ecb41d4562deec2a1c7c148405 my-package_1.2.3_all.deb

 0cdac6bc757dd2b78bacc13253_my-package_1.2.3_all.deb

MD5

For each uploaded Debian package a MD5 checksum is computed. This checksum is used to create a file tree where each package will reside, with following hierarchy

Note: MD5 is only legacy layout. Its support is limited to 'read' files from the pool, it never puts files this way for new package files.

Directory and sub-directories structure

  • 1st and 2nd characters of MD5 checksum as sub-directory name of rootDir/pool directory
  • 3rd and 4th characters of MD5 chacksum as sub-directory name of the former

ex:

md5sum feea3c0c3e823615bf2d417b052a96b4 my-package_1.2.3_all.deb

${rootDir}/pool # rootDir defined in aptly.conf
└── fe
    └── ea

Filename

Uploaded Debian is stored as-is and not renamed.

Example

${rootDir}/pool # rootDir defined in aptly.conf
├── 00
│   ├── 25
│   │   └── yet_another_package-0.6.0_all.deb
│   ├── 60
│   ├── 97
│   │   └── 80ced73165f92fea490f2561a7c4_my-package_0.0.1_all.deb
│   ├── 6e 
│   │   └── 0cdac6bc757dd2b78bacc13253_my-package_1.2.3_all.deb # sha256sum 476e0cdac6bc757dd2b78bacc1325323b09c45ecb41d4562deec2a1c7c148405
│   └── db
│       └── yet_another_package-0.5.8_all.deb # md5sum 00db7ada61aa28a6931267f1714cbb15
...
├── 2a                                                                                                                
│   ├── 10                                                                                                            
│   │   └── yet_another_package-0.5.9_all.deb
│   ├── 64
│   │   └── 80ced73165f92fea490f2561a7c4_my-other-package_2.3.2_amd64.deb
│   ├── 4c                                                                                                            
│   ├── 5c                                                                                                            
│   │   └── yet_another_package-0.6.1_all.deb
│   ├── 77                                                                                                            
│   ├── b5                                                                                                            
│   │   └── 4b2eb349236cf5c4af7eca68a43b_my-package_0.2.0_amd64.deb
...
└── ff
    ├── 4c                                                                                                            
    ├── 5a                                                                                                            
    │   └── 8868dd8661bbe25c51bdd9b2d25c_my-package_0.2.0_amd64.deb                                          
    └── dc