TLDR, my main questions:
Does the tar format itself support checksums?
How to create a tar (or even better, tar.gz file) with the tar format built-in checksum?
Long: Before you ask why I am asking or suggest alternatives...
Why do I think that such a thing might exist?
VirtualBox source code file tarvfs.cpp indicates that. Here is the relevant code snippet: [1] [2]
- What I am looking for: Does any tar format support built-in checksums?
- What I am not looking for:
- A workaround using sha256sums or similar / a file added to the archive itself. Won't work for this use case.
vboxmanage modifymedium --compact/zerofreesince already using this.
- Why do I care?
Trying to import a VirtualBox ova that I created manually using the command line, I am getting a VERR_TAR_BAD_CHKSUM_FIELD
tar --gzip --create --verbose --directory=/home/user/temp --file empty.ova temp-disk001.vmdk temp-disk002.vmdk temp.mf temp.ovf
temp-disk001.vmdk temp-disk002.vmdk temp.mf temp.ovf
vboxmanage import empty.ova ; echo $?
Progress state: VBOX_E_IPRT_ERROR
VBoxManage: error: Appliance read failed
VBoxManage: error: Error reading OVA '/home/user/temp/empty.ova' (VERR_TAR_BAD_CHKSUM_FIELD)
VBoxManage: error: Details: code VBOX_E_IPRT_ERROR (0x80bb0005), component ApplianceWrap, interface IAppliance
VBoxManage: error: Context: "RTEXITCODE handleImportAppliance(HandlerArg*)" at line 471 of file VBoxManageAppliance.cpp
If I drop --gzip from tar parameters it's functional.
tar --create --verbose --directory=/home/user/temp --file empty.ova temp-disk001.vmdk temp-disk002.vmdk temp.mf temp.ovf
Then import works.
vboxmanage import empty.ova ; echo $?
[...] 0
- Why am I not using
vboxmanage export? Because it creates a tar file without encryption, does not gzip compression. - What my main goal? A gzip compressed
.ovaVirtualBox appliances that can be imported directly into VirtualBox without an additional.tar.gz. - Aren't VirtualBox ova's compressed anyhow? No. These are just tar files. Not gzip or xz compressed archives.
- But the
.vmdkfiles in.ovafiles are compressed? Yes, but the compression is limited. By using an additional.tar.gzwith maximum compression, the.ovagets smaller. - Why not use an additional
.tar.gzfile? Compression, speed, usability, simplicity. - Why do you think that VirtualBox can import
.ovas that are actually.tar.gzfiles? Because VirtualBox source file ApplianceImplImport.cpp mentions gzip several times. - Why am I mentioning all of this? Because most questions that I've seen go into direction of
vboxmanage modifymedium --compact/zerofree. Therefore I wanted to be very specific here: An.ovafile that is gzip compressed.
[1]
/**
* Validates the TAR header.
*
* @returns VINF_SUCCESS if valid, VERR_TAR_ZERO_HEADER if all zeros, and
* the appropriate VERR_TAR_XXX otherwise.
* @param pTar The TAR header.
* @param penmType Where to return the type of header on success.
*/
static int rtZipTarHdrValidate(PCRTZIPTARHDR pTar, PRTZIPTARTYPE penmType)
{
/*
* Calc the checksum first since this enables us to detect zero headers.
*/
int32_t i32ChkSum;
int32_t i32ChkSumSignedAlt;
if (rtZipTarCalcChkSum(pTar, &i32ChkSum, &i32ChkSumSignedAlt))
return VERR_TAR_ZERO_HEADER;
/*
* Read the checksum field and match the checksums.
*/
int64_t i64HdrChkSum;
int rc = rtZipTarHdrFieldToNum(pTar->Common.chksum, sizeof(pTar->Common.chksum), true /*fOctalOnly*/, &i64HdrChkSum);
if (RT_FAILURE(rc))
return VERR_TAR_BAD_CHKSUM_FIELD;
if ( i32ChkSum != i64HdrChkSum
&& i32ChkSumSignedAlt != i64HdrChkSum) /** @todo test this */
return VERR_TAR_CHKSUM_MISMATCH;
[2]
/*
* Copyright (C) 2010-2020 Oracle Corporation
*
* This file is part of VirtualBox Open Source Edition (OSE), as
* available from http://www.virtualbox.org. This file is free software;
* you can redistribute it and/or modify it under the terms of the GNU
* General Public License (GPL) as published by the Free Software
* Foundation, in version 2 as it comes in the "COPYING" file of the
* VirtualBox OSE distribution. VirtualBox OSE is distributed in the
* hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
*
* The contents of this file may alternatively be used under the terms
* of the Common Development and Distribution License Version 1.0
* (CDDL) only, as it comes in the "COPYING.CDDL" file of the
* VirtualBox OSE distribution, in which case the provisions of the
* CDDL are applicable instead of those of the GPL.
*
* You may elect to license modified versions of this file under the
* terms and conditions of either the GPL or the CDDL or both.
*/
Update 1:
- tar checksums but not details: https://www.math.utah.edu/docs/info/tar_8.html#SEC85
- gzip checksums: Does gzip add integrity/crc check to a .tar?
- star
artype=crc(Thanks to commentor @artem-s-tashkinov)