I am trying to build all of the dependencies for Weasyprint 52.5 for AWS Lambda. Despite being a python library, Weasyprint has a large number of OS dependencies.
I'm creating a lambda layer by downloading all of the necessary rpms from yum and extracting the shared libraries. It nearly works but I can't seem to get around this error with libgio.
The OS is Amazon Linux 2, which uses yum for package management and seems to match up with CentOS 7.
This is the yum call command:
yumdownloader libffi pixman freetype fontconfig libglvnd libglvnd-glx libglvnd-egl mesa-libglapi libpng libxcb libXrender libX11 libXext libXau libXdamage libXfixes libXxf86vm expat libuuid libxshmfence libdrm libwayland-client libwayland-server glib2 fribidi libthai harfbuzz graphite2 gdk-pixbuf2 cairo pango
Which pulls these package versions:
cairo-1.15.12-4.amzn2.x86_64
expat-2.1.0-12.amzn2.x86_64
fontconfig-2.13.0-4.3.amzn2.x86_64
freetype-2.8-14.amzn2.1.x86_64
fribidi-1.0.2-1.amzn2.1.x86_64
gdk-pixbuf2-2.36.12-3.amzn2.x86_64
glib2-2.56.1-9.amzn2.0.1.x86_64
graphite2-1.3.10-1.amzn2.0.2.x86_64
harfbuzz-1.7.5-2.amzn2.x86_64
libdrm-2.4.97-2.amzn2.x86_64
libffi-3.0.13-18.amzn2.0.2.x86_64
libglvnd-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
libglvnd-egl-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
libglvnd-glx-1.0.1-0.1.git5baa1e5.amzn2.0.1.x86_64
libpng-1.5.13-8.amzn2.x86_64
libthai-0.1.14-9.amzn2.0.2.x86_64
libuuid-2.30.2-2.amzn2.0.4.x86_64
libwayland-client-1.17.0-1.amzn2.x86_64
libwayland-server-1.17.0-1.amzn2.x86_64
libX11-1.6.7-3.amzn2.0.2.x86_64
libXau-1.0.8-2.1.amzn2.0.2.x86_64
libxcb-1.12-1.amzn2.0.2.x86_64
libXdamage-1.1.4-4.1.amzn2.0.2.x86_64
libXext-1.3.3-3.amzn2.0.2.x86_64
libXfixes-5.0.3-1.amzn2.0.2.x86_64
libXrender-0.9.10-1.amzn2.0.2.x86_64
libxshmfence-1.2-1.amzn2.0.2.x86_64
libXxf86vm-1.1.4-1.amzn2.0.2.x86_64
mesa-libglapi-18.3.4-5.amzn2.0.1.x86_64
pango-1.42.4-4.amzn2.x86_64
pixman-0.34.0-1.amzn2.0.2.x86_64
Running rpmdev-extract to extract the rpms results in these shared libraries:
libcairo-script-interpreter.so.2.11512.0
libcairo.so.2.11512.0
libdrm_amdgpu.so.1.0.0
libdrm_intel.so.1.0.0
libdrm_nouveau.so.2.0.0
libdrm_radeon.so.1.0.1
libdrm.so.2.4.0
libEGL.so.1.1.0
libexpat.so.1.6.0
libffi.so.6.0.1
libfontconfig.so.1.11.1
libfreetype.so.6.14.0
libfribidi.so.0.4.0
libgdk_pixbuf-2.0.so.0.3612.0
libgdk_pixbuf_xlib-2.0.so.0.3612.0
libgio-2.0.so.0.5600.1
libglapi.so.0.0.0
libGLdispatch.so.0.0.0
libglib-2.0.so.0.5600.1
libGL.so.1.7.0
libGLX.so.0.0.0
libgmodule-2.0.so.0.5600.1
libgobject-2.0.so.0.5600.1
libgraphite2.so.3.0.1
libgthread-2.0.so.0.5600.1
libharfbuzz.so.0.10705.0
libkms.so.1.0.0
libpango-1.0.so.0.4200.3
libpangocairo-1.0.so.0.4200.3
libpangoft2-1.0.so.0.4200.3
libpangoxft-1.0.so.0.4200.3
libpixman-1.so.0.34.0
libpng15.so.15
libpng15.so.15.13.0
libthai.so.0.1.6
libuuid.so.1.3.0
libwayland-client.so.0.3.0
libwayland-server.so.0.1.0
libX11.so.6.3.0
libX11-xcb.so.1.0.0
libXau.so.6.0.0
libxcb-composite.so.0.0.0
libxcb-damage.so.0.0.0
libxcb-dpms.so.0.0.0
libxcb-dri2.so.0.0.0
libxcb-dri3.so.0.0.0
libxcb-glx.so.0.0.0
libxcb-present.so.0.0.0
libxcb-randr.so.0.1.0
libxcb-record.so.0.0.0
libxcb-render.so.0.0.0
libxcb-res.so.0.0.0
libxcb-screensaver.so.0.0.0
libxcb-shape.so.0.0.0
libxcb-shm.so.0.0.0
libxcb.so.1.1.0
libxcb-sync.so.1.0.0
libxcb-xevie.so.0.0.0
libxcb-xf86dri.so.0.0.0
libxcb-xfixes.so.0.0.0
libxcb-xinerama.so.0.0.0
libxcb-xinput.so.0.1.0
libxcb-xkb.so.1.0.0
libxcb-xselinux.so.0.0.0
libxcb-xtest.so.0.0.0
libxcb-xvmc.so.0.0.0
libxcb-xv.so.0.0.0
libXdamage.so.1.1.0
libXext.so.6.4.0
libXfixes.so.3.1.0
libXrender.so.1.3.0
libxshmfence.so.1.0.0
libXxf86vm.so.1.0.0
When Weasyprint calls ffi.dlopen("libfontconfig.so.1"), I get an OSError:
cannot load library 'libfontconfig.so.1': /opt/lib/libgio-2.0.so.0: undefined symbol: g_free
I have tried downgrading some of the key packages to the other versions available on yum, but with no success. Specifically, I have tried downgrading the following:
fontconfig-2.10.95
glib2-2.50.3
cairo-1.14.8
pango-1.40.4
but this results in the same error.
What is causing the undefined symbol error? Is there any workaround that will resolve this error?
UPDATE
I'm now wondering if this is related to AWS Lambda specifically.
I am building the dependencies on an Amazon2 Docker instance however I cannot replicate the problem in my Docker container. For example, here is my current Dockerfile, run on an Amazon2 EC2 instance:
Note that this Dockerfile uses a slightly modified version of Weasyprint 52.5 where I've added some extra error logging. I also added some code to force ffi to load libraries specifically from /opt/lib:
https://github.com/Kozea/WeasyPrint/compare/v52.5...MasonryHQ:temp?expand=1
When I build this Docker container, I encounter no errors. However when I zip up my /opt/lib directory and upload it to Lambda as a layer, I still get the error on Lambda as soon as I try to import Weasyprint. See:
https://sentry.io/share/issue/1d2d824e950346e9bcc368f8abfd01f0/
Is it possible some libraries in Docker are still not loading from /opt/lib despite my changes to Weasyprint? Or is Lambda just behaving differently than the Docker container?
