2

TLDR, my main questions:

Does the tar format itself support checksums?

How to create a tar (or even better, tar.gz file) with the tar format built-in checksum?


Long: Before you ask why I am asking or suggest alternatives...

Why do I think that such a thing might exist?

VirtualBox source code file tarvfs.cpp indicates that. Here is the relevant code snippet: [1] [2]

  • What I am looking for: Does any tar format support built-in checksums?
  • What I am not looking for:
  • Why do I care?

Trying to import a VirtualBox ova that I created manually using the command line, I am getting a VERR_TAR_BAD_CHKSUM_FIELD

tar --gzip --create --verbose --directory=/home/user/temp --file empty.ova temp-disk001.vmdk temp-disk002.vmdk temp.mf temp.ovf

temp-disk001.vmdk temp-disk002.vmdk temp.mf temp.ovf

vboxmanage import empty.ova ; echo $?
Progress state: VBOX_E_IPRT_ERROR
VBoxManage: error: Appliance read failed
VBoxManage: error: Error reading OVA '/home/user/temp/empty.ova' (VERR_TAR_BAD_CHKSUM_FIELD)
VBoxManage: error: Details: code VBOX_E_IPRT_ERROR (0x80bb0005), component ApplianceWrap, interface IAppliance
VBoxManage: error: Context: "RTEXITCODE handleImportAppliance(HandlerArg*)" at line 471 of file VBoxManageAppliance.cpp

If I drop --gzip from tar parameters it's functional.

tar    --create    --verbose      --directory=/home/user/temp    --file empty.ova    temp-disk001.vmdk  temp-disk002.vmdk  temp.mf  temp.ovf

Then import works.

vboxmanage import empty.ova ; echo $?

[...] 0

  • Why am I not using vboxmanage export? Because it creates a tar file without encryption, does not gzip compression.
  • What my main goal? A gzip compressed .ova VirtualBox appliances that can be imported directly into VirtualBox without an additional .tar.gz.
  • Aren't VirtualBox ova's compressed anyhow? No. These are just tar files. Not gzip or xz compressed archives.
  • But the .vmdk files in .ova files are compressed? Yes, but the compression is limited. By using an additional .tar.gz with maximum compression, the .ova gets smaller.
  • Why not use an additional .tar.gz file? Compression, speed, usability, simplicity.
  • Why do you think that VirtualBox can import .ovas that are actually .tar.gz files? Because VirtualBox source file ApplianceImplImport.cpp mentions gzip several times.
  • Why am I mentioning all of this? Because most questions that I've seen go into direction of vboxmanage modifymedium --compact / zerofree. Therefore I wanted to be very specific here: An .ova file that is gzip compressed.

[1]

/**
 * Validates the TAR header.
 *
 * @returns VINF_SUCCESS if valid, VERR_TAR_ZERO_HEADER if all zeros, and
 *          the appropriate VERR_TAR_XXX otherwise.
 * @param   pTar                The TAR header.
 * @param   penmType            Where to return the type of header on success.
 */
static int rtZipTarHdrValidate(PCRTZIPTARHDR pTar, PRTZIPTARTYPE penmType)
{
    /*
     * Calc the checksum first since this enables us to detect zero headers.
     */
    int32_t i32ChkSum;
    int32_t i32ChkSumSignedAlt;
    if (rtZipTarCalcChkSum(pTar, &i32ChkSum, &i32ChkSumSignedAlt))
        return VERR_TAR_ZERO_HEADER;

    /*
     * Read the checksum field and match the checksums.
     */
    int64_t i64HdrChkSum;
    int rc = rtZipTarHdrFieldToNum(pTar->Common.chksum, sizeof(pTar->Common.chksum), true /*fOctalOnly*/, &i64HdrChkSum);
    if (RT_FAILURE(rc))
        return VERR_TAR_BAD_CHKSUM_FIELD;
    if (   i32ChkSum          != i64HdrChkSum
        && i32ChkSumSignedAlt != i64HdrChkSum) /** @todo test this */
        return VERR_TAR_CHKSUM_MISMATCH;

[2]

/*
 * Copyright (C) 2010-2020 Oracle Corporation
 *
 * This file is part of VirtualBox Open Source Edition (OSE), as
 * available from http://www.virtualbox.org. This file is free software;
 * you can redistribute it and/or modify it under the terms of the GNU
 * General Public License (GPL) as published by the Free Software
 * Foundation, in version 2 as it comes in the "COPYING" file of the
 * VirtualBox OSE distribution. VirtualBox OSE is distributed in the
 * hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
 *
 * The contents of this file may alternatively be used under the terms
 * of the Common Development and Distribution License Version 1.0
 * (CDDL) only, as it comes in the "COPYING.CDDL" file of the
 * VirtualBox OSE distribution, in which case the provisions of the
 * CDDL are applicable instead of those of the GPL.
 *
 * You may elect to license modified versions of this file under the
 * terms and conditions of either the GPL or the CDDL or both.
 */

Update 1:

adrelanos
  • 1,786
  • 7
  • 29
  • 56
  • You mention that you think that VirtualBox can import `.ova` files that are actually `.tar.gz` files "because VirtualBox source file ApplianceImplImport.cpp mentions gzip several times", but are you actually quite sure of that? At the very least, it seems to be an undocumented feature. Also (I'm not to familiar with VirtualBox), could you not just do something along the lines of `gunzip -c empty.ova | vboxmanage import -`? – Malte Skoruppa Sep 09 '22 at 23:05
  • That would work but is unwanted. `.ova` files are associated with VirtualBox. When double clicking an `.ova` file it is opened by the correct handler, VirtualBox. I'd like to keep it that way for better usability. Undocumented VirtualBox feature, most likely yes. This isn't really a VirtualBox question. For the VirtualBox part, I am asking in VirtualBox forums. https://forums.virtualbox.org/viewtopic.php?f=1&t=107051 The mention of VirtualBox comes from that VirtualBox seems to check a tar checksum, which I haven't found elsewhere and haven o idea how it could be created. Hence, asked here. – adrelanos Sep 10 '22 at 08:32
  • 1
    `star artype=crc` – Artem S. Tashkinov Sep 10 '22 at 09:42

0 Answers0