cloud-init will not run user-data scripts when /var filesystem is mounted with the noexec flag

This bug was originally filed in Launchpad as [LP: #1839899](https://bugs.launchpad.net/cloud-init/+bug/1839899)

Launchpad details

affected_projects = []

assignee = None

assignee_name = None

date_closed = None

date_created = 2019-08-13T00:57:41.794994+00:00

date_fix_committed = None

date_fix_released = None

id = 1839899

importance = medium

is_complete = False

lp_url = https://bugs.launchpad.net/cloud-init/+bug/1839899

milestone = None

owner = nemusupport

owner_name = Nému Support

private = False

status = triaged

submitter = nemusupport

submitter_name = Nému Support

tags = ['aws', 'rhel', 'selinux']

duplicates = []

_Launchpad user **Nému Support(nemusupport)** wrote on 2019-08-13T00:57:41.794994+00:00_

Cloud Vendor: Amazon AWS
Platform: RHEL7.6
Cloud-Init: cloud-init-18.5-3.el7.x86_64
Kernel: 3.10.0-1062.el7.x86_64
SELinux: selinux-policy-targeted-3.13.1-252.el7.1.noarch

--

We have identified that having the "noexec" flag set on the /var filesystem causes cloud-init to fail running user-data scripts. This is a security requirement mandated by STIG policies that we're purposefully trying to meet for Federal systems.

The affected code is in:

/usr/lib/python2.7/site-packages/cloudinit/util.py

Under the function:

runparts()

The system checks for access to the executable using the following line:

        if os.path.isfile(exe_path) and os.access(exe_path, os.X_OK):
                                    ## ^^^^^^^^^

While the file is executable, the "noexec" flag on the filesystem causes os.access() to report False, which cancels the execution of the user-data script.

To reproduce the problem:

- Create new filesystem
- Move /var files to new filesystem
- Add /var to fstab with the "noexec" option
- Mount new /var filesystem
- Run cloud-init init
- Run cloud-init modules -m final
- Observe that the cloud-init scripts do not run

Note that the files in /var/lib/cloud/instances/*/scripts/ are executable (mode 0755 or 0700)

And that when trying to execute the file, you will get Error 13: Permission denied.

--

Possible fixes:

- Search for marker on the first line of the file (#!) and add the requested shell as exe_prefix (as stated above)
- Move /var/lib/cloud (or a portion thereof) to a different filesystem path and symlink it to original path

We have tested the second workaround and it seems to help:

# cloud-init clean
# rm -Rf /var/lib/cloud
# mkdir -p /etc/cloud/runtime
# ln -s /etc/cloud/runtime /var/lib/cloud
# restorecon -rv /var/lib/cloud

After this, user-data scripts appear to execute.