Skip to content

pyrex.ini: add /tmp to default [run]bind#105

Open
JoshuaWatt wants to merge 1 commit intogarmin:masterfrom
JoshuaWatt:bind-tmp
Open

pyrex.ini: add /tmp to default [run]bind#105
JoshuaWatt wants to merge 1 commit intogarmin:masterfrom
JoshuaWatt:bind-tmp

Conversation

@JoshuaWatt
Copy link
Collaborator

I observed anomalously high disk I/O on my root filesystem during an OpenEmbedded build which used pyrex. (In fact, no rootfs I/O should generally be happening on this machine during an OpenEmbedded build; both the sources and TMPDIR are on other disks, and /tmp is a tmpfs.) After some judicious debugging with fatrace(1) I noticed that virtually all of the I/O was happening against, e.g.,
/var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o.

It appears that under pyrex, bitbake will use the /tmp in the container overlay, not the system's /tmp. I believe that this is a mistake: pyrex containers should always use /tmp directly through a bind mount. My reasoning is as follows.

  • On systems where /tmp is a tmpfs, using it will effect a major performance improvement, and potentially reduce root filesystem wear.
  • By using a tmpfs instead of the root filesystem, this change cannot introduce any OOM that wouldn't have happened anyway without pyrex.
  • The use of /tmp in recipes must already be robust in the presence of parallel builds; any collision between simultaneous builds under different containers represents a fault in the recipe, not in the bind-mounting.
  • No security problems can occur through bind-mounting /tmp that would not have happened anyway when building outside of pyrex.

Closes #95

I observed anomalously high disk I/O on my root filesystem during an
OpenEmbedded build which used pyrex. (In fact, *no* rootfs I/O should generally
be happening on this machine during an OpenEmbedded build; both the sources and
TMPDIR are on other disks, and `/tmp` is a tmpfs.) After some judicious debugging
with fatrace(1) I noticed that virtually all of the I/O was happening against,
e.g.,
`/var/lib/docker/overlay2/dc702bbbb061fbbf13b9d06c36d8ccee398257070e6c0455ff209b656b2db2f3/diff/tmp/ccuwDoBf.o`.

It appears that under pyrex, bitbake will use the `/tmp` in the container
overlay, not the system's `/tmp`. I believe that this is a mistake: pyrex
containers should *always* use `/tmp` directly through a bind mount. My
reasoning is as follows.

- On systems where `/tmp` is a tmpfs, using it will effect a major performance
  improvement, and potentially reduce root filesystem wear.
- By using a tmpfs instead of the root filesystem, this change cannot introduce
  any OOM that wouldn't have happened anyway without pyrex.
- The use of `/tmp` in recipes must already be robust in the presence of
  parallel builds; any collision between simultaneous builds under different
  containers represents a fault in the recipe, not in the bind-mounting.
- No security problems can occur through bind-mounting `/tmp` that would not
  have happened anyway when building outside of pyrex.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants