audit watch rules and docker containers

[Rakesh]

Hello Auditd'ers,
I am running a privileged container with pid, net, uts space shared with the host. The need is to be able to set file watch rules from the container say 
-k /etc -p rw -k containter_rule
and then look for read/write access to files/directories in /var/log/audit/*.
What I am finding is there are no watch events being logged
If I set the same audit watch rule from the host (and not being in the privileged container) I am able to get audit events
Using nsenter to switch namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule) does not help either
I suspect the mnt namespace is different which is causing this oddity in behavior
looking at container process namespace -
***@ubuntu-16:~/audit$ sudo ls -latr  /proc/26050/ns[sudo] password for test:total 0dr-xr-xr-x 9 root root 0 Mar  2 16:58 ..dr-x--x--x 2 root root 0 Mar  2 17:46 .lrwxrwxrwx 1 root root 0 Mar  2 17:46 uts -> uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 17:46 user -> user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 17:46 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 17:46 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 17:46 mnt -> mnt:[4026532517]lrwxrwxrwx 1 root root 0 Mar  2 17:46 ipc -> ipc:[4026532518]lrwxrwxrwx 1 root root 0 Mar  2 17:46 cgroup -> cgroup:[4026531835]
looking at init process namespace -

***@ubuntu-16:~/audit$ sudo ls -latr  /proc/1/nstotal 0dr-xr-xr-x 9 root root 0 Mar  2 10:37 ..lrwxrwxrwx 1 root root 0 Mar  2 10:38 mnt -> mnt:[4026531840]dr-x--x--x 2 root root 0 Mar  2 10:38 .lrwxrwxrwx 1 root root 0 Mar  2 16:47 uts -> uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 16:47 user -> user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 16:47 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 16:47 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 16:47 ipc -> ipc:[4026531839]lrwxrwxrwx 1 root root 0 Mar  2 16:47 cgroup -> cgroup:[4026531835]
Can someone suggest on how to make this work.
Thanks,Rakesh

[Rakesh]

Hello Auditd'ers,



I am running a privileged container with pid, net, uts space shared with the host. The need is to be able to set file watch rules from the container say 
-k /etc -p rw -k containter_rule
and then look for read/write access to files/directories in /var/log/audit/*.
What I am finding is there are no watch events being logged
If I set the same audit watch rule from the host (and not being in the privileged container) I am able to get audit events
Using nsenter to switch namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule) does not help either
I suspect the mnt namespace is different which is causing this oddity in behavior
looking at container process namespace -
***@ubuntu-16:~/audit$ sudo ls -latr  /proc/26050/ns[sudo] password for test:total 0dr-xr-xr-x 9 root root 0 Mar  2 16:58 ..dr-x--x--x 2 root root 0 Mar  2 17:46 .lrwxrwxrwx 1 root root 0 Mar  2 17:46 uts -> uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 17:46 user -> user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 17:46 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 17:46 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 17:46 mnt -> mnt:[4026532517]lrwxrwxrwx 1 root root 0 Mar  2 17:46 ipc -> ipc:[4026532518]lrwxrwxrwx 1 root root 0 Mar  2 17:46 cgroup -> cgroup:[4026531835]
looking at init process namespace -

***@ubuntu-16:~/audit$ sudo ls -latr  /proc/1/nstotal 0dr-xr-xr-x 9 root root 0 Mar  2 10:37 ..lrwxrwxrwx 1 root root 0 Mar  2 10:38 mnt -> mnt:[4026531840]dr-x--x--x 2 root root 0 Mar  2 10:38 .lrwxrwxrwx 1 root root 0 Mar  2 16:47 uts -> uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 16:47 user -> user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 16:47 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 16:47 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 16:47 ipc -> ipc:[4026531839]lrwxrwxrwx 1 root root 0 Mar  2 16:47 cgroup -> cgroup:[4026531835]
Can someone please suggest with some thoughts on how to make this work.
Thanks,Rakesh

[Steve Grubb]

On Sat, 3 Mar 2018 08:52:04 +0000 (UTC)

Post by Rakesh
Hello Auditd'ers,
I am running a privileged container with pid, net, uts space shared
with the host. The need is to be able to set file watch rules from
the container say -k /etc -p rw -k containter_rule and then look for
read/write access to files/directories in /var/log/audit/*.

Container support is just now being implemented.

https://www.redhat.com/archives/linux-audit/2018-March/msg00004.html

So, there is no good way right now to make this work like you would
expect it.

-Steve

Post by Rakesh
What I am
finding is there are no watch events being logged If I set the same
audit watch rule from the host (and not being in the privileged
container) I am able to get audit events Using nsenter to switch
namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule)
does not help either I suspect the mnt namespace is different which
is causing this oddity in behavior looking at container process
-latr  /proc/26050/ns[sudo] password for test:total 0dr-xr-xr-x 9
root root 0 Mar  2 16:58 ..dr-x--x--x 2 root root 0 Mar  2
17:46 .lrwxrwxrwx 1 root root 0 Mar  2 17:46 uts ->
uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 17:46 user ->
user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 17:46 pid ->
pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 17:46 net ->
net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 17:46 mnt ->
mnt:[4026532517]lrwxrwxrwx 1 root root 0 Mar  2 17:46 ipc ->
ipc:[4026532518]lrwxrwxrwx 1 root root 0 Mar  2 17:46 cgroup ->
cgroup:[4026531835] looking at init process namespace -
root root 0 Mar  2 10:37 ..lrwxrwxrwx 1 root root 0 Mar  2 10:38 mnt
-> mnt:[4026531840]dr-x--x--x 2 root root 0 Mar  2 10:38 .lrwxrwxrwx
1 root root 0 Mar  2 16:47 uts -> uts:[4026531838]lrwxrwxrwx 1 root
root 0 Mar  2 16:47 user -> user:[4026531837]lrwxrwxrwx 1 root root 0
Mar  2 16:47 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2
16:47 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 16:47
ipc -> ipc:[4026531839]lrwxrwxrwx 1 root root 0 Mar  2 16:47 cgroup
-> cgroup:[4026531835] Can someone please suggest with some thoughts
on how to make this work. Thanks,Rakesh

[Rakesh]

Hi Steve,
Thanks for taking the time to look at it. I have been following the conversation on adding container support to audit, however I am not looking for container id in the event.
I did some more tests and find it works as expected for syscalls -
-a always,exit -F arch=b64 -S connect -F exit!=-ENOENT -F key=connect

and the audit event in log is -
arch=c000003e syscall=42 success=yes exit=0 a0=1 a1=5562d1bb40f8 a2=16 a3=7ffd9db76460 items=1 ppid=2 pid=60470 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="systemd-cgroups" exe="/lib/systemd/systemd-cgroups-agent" key="connect"

Bit it's the watch events which are not working.

Thanks,Rakesh

From: Steve Grubb <***@redhat.com>
To: Rakesh <***@yahoo.com>
Cc: "linux-***@redhat.com" <linux-***@redhat.com>
Sent: Sunday, March 4, 2018 5:00 AM
Subject: Re: audit watch rules and docker containers

On Sat, 3 Mar 2018 08:52:04 +0000 (UTC)

Post by Rakesh
Hello Auditd'ers,
I am running a privileged container with pid, net, uts space shared
with the host. The need is to be able to set file watch rules from
the container say -k /etc -p rw -k containter_rule and then look for
read/write access to files/directories in /var/log/audit/*.

Container support is just now being implemented.

https://www.redhat.com/archives/linux-audit/2018-March/msg00004.html

So, there is no good way right now to make this work like you would
expect it.

-Steve

Post by Rakesh
What I am
finding is there are no watch events being logged If I set the same
audit watch rule from the host (and not being in the privileged
container) I am able to get audit events Using nsenter to switch
namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule)
does not help either I suspect the mnt namespace is different which
is causing this oddity in behavior looking at container process
-latr  /proc/26050/ns[sudo] password for test:total 0dr-xr-xr-x 9
root root 0 Mar  2 16:58 ..dr-x--x--x 2 root root 0 Mar  2
17:46 .lrwxrwxrwx 1 root root 0 Mar  2 17:46 uts ->
uts:[4026531838]lrwxrwxrwx 1 root root 0 Mar  2 17:46 user ->
user:[4026531837]lrwxrwxrwx 1 root root 0 Mar  2 17:46 pid ->
pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2 17:46 net ->
net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 17:46 mnt ->
mnt:[4026532517]lrwxrwxrwx 1 root root 0 Mar  2 17:46 ipc ->
ipc:[4026532518]lrwxrwxrwx 1 root root 0 Mar  2 17:46 cgroup ->
cgroup:[4026531835] looking at init process namespace -
root root 0 Mar  2 10:37 ..lrwxrwxrwx 1 root root 0 Mar  2 10:38 mnt
-> mnt:[4026531840]dr-x--x--x 2 root root 0 Mar  2 10:38 .lrwxrwxrwx
1 root root 0 Mar  2 16:47 uts -> uts:[4026531838]lrwxrwxrwx 1 root
root 0 Mar  2 16:47 user -> user:[4026531837]lrwxrwxrwx 1 root root 0
Mar  2 16:47 pid -> pid:[4026531836]lrwxrwxrwx 1 root root 0 Mar  2
16:47 net -> net:[4026531957]lrwxrwxrwx 1 root root 0 Mar  2 16:47
ipc -> ipc:[4026531839]lrwxrwxrwx 1 root root 0 Mar  2 16:47 cgroup
-> cgroup:[4026531835] Can someone please suggest with some thoughts
on how to make this work. Thanks,Rakesh 

[Steve Grubb]

On Mon, 5 Mar 2018 03:06:44 +0000 (UTC)

Post by Rakesh
Hi Steve,
Thanks for taking the time to look at it. I have been following the
conversation on adding container support to audit, however I am not
looking for container id in the event. I did some more tests and find
it works as expected for syscalls - -a always,exit -F arch=b64 -S
connect -F exit!=-ENOENT -F key=connect
and the audit event in log is -
arch=c000003e syscall=42 success=yes exit=0 a0=1 a1=5562d1bb40f8
a2=16 a3=7ffd9db76460 items=1 ppid=2 pid=60470 auid=4294967295 uid=0
gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none)
ses=4294967295 comm="systemd-cgroups"
exe="/lib/systemd/systemd-cgroups-agent" key="connect"
Bit it's the watch events which are not working.

Watches are a convenience that changes a human path into a device and
inode. That is really what is watched. I think that if you have a watch
on /etc/passwd, and a container has its own /etc/passwd, then you will
have a different inode if not device.

Hopefully this is being taken into account with the redesign or at
least the ability to express that you want them all somehow.

-Steve

[Richard Guy Briggs]

Post by Rakesh
Hello Auditd'ers,

Hi Rakesh,
(I see, with difficulty, that your output is well-formatted in the HTML
attachment, but that isn't useful. Please shut off HTML message
formatting and ensure that it looks right in plain text. Also, please
use "ls -l" so it sorts in a meaningful order for comparison.)

Post by Rakesh
I am running a privileged container with pid, net, uts space shared with the host. The need is to be able to set file watch rules from the container say 
-k /etc -p rw -k containter_rule
and then look for read/write access to files/directories in /var/log/audit/*.
What I am finding is there are no watch events being logged
If I set the same audit watch rule from the host (and not being in the privileged container) I am able to get audit events
Using nsenter to switch namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule) does not help either
I suspect the mnt namespace is different which is causing this oddity in behavior
looking at container process namespace -
looking at init process namespace -

After decoding your jumbled mess of output due to HTML and ls options
choices, the mount namespaces are different, which would completely
explain the problem.

Post by Rakesh
Can someone please suggest with some thoughts on how to make this work.

The pending container support mentioned by Steve is not yet complete and
some more of the coming changes may help with your issue, but start by
understanding that you are examining different filesystems with your
rules above.

Post by Rakesh
Thanks,Rakesh

- RGB

--
Richard Guy Briggs <***@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

[Rakesh]

Hi Richard,


Thanks for reviewing the email and my apologies for the formatting issue. This response corrects that.

I looked at Steve's response (with the embedded link) and have also followed your presentation on youtube however I am not clear on the proposed change(s) which will allow the mnt space to be shared between the host and the privileged container. Is this use case even being considered?
Thanks,
Rakesh



________________________________
From: Richard Guy Briggs <***@redhat.com>
To: Rakesh <***@yahoo.com>
Cc: "linux-***@redhat.com" <linux-***@redhat.com>
Sent: Sunday, March 4, 2018 11:14 PM
Subject: Re: audit watch rules and docker containers

Post by Rakesh
Hello Auditd'ers,

Hi Rakesh,
(I see, with difficulty, that your output is well-formatted in the HTML
attachment, but that isn't useful. Please shut off HTML message
formatting and ensure that it looks right in plain text. Also, please
use "ls -l" so it sorts in a meaningful order for comparison.)

Post by Rakesh
I am running a privileged container with pid, net, uts space shared with the host. The need is to be able to set file watch rules from the container say
-k /etc -p rw -k containter_rule
and then look for read/write access to files/directories in /var/log/audit/*.
What I am finding is there are no watch events being logged
If I set the same audit watch rule from the host (and not being in the privileged container) I am able to get audit events
Using nsenter to switch namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule) does not help either
I suspect the mnt namespace is different which is causing this oddity in behavior
looking at container process namespace -
looking at init process namespace -

After decoding your jumbled mess of output due to HTML and ls options
choices, the mount namespaces are different, which would completely
explain the problem.

Post by Rakesh
Can someone please suggest with some thoughts on how to make this work.

The pending container support mentioned by Steve is not yet complete and
some more of the coming changes may help with your issue, but start by
understanding that you are examining different filesystems with your
rules above.

Post by Rakesh
Thanks,Rakesh

- RGB

--
Richard Guy Briggs <***@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

[Richard Guy Briggs]

Post by Rakesh
Hi Richard,
Thanks for reviewing the email and my apologies for the formatting
issue. This response corrects that.

(I don't see any format correction in this response.)

Post by Rakesh
I looked at Steve's response (with the embedded link) and have also
followed your presentation on youtube however I am not clear on the
proposed change(s) which will allow the mnt space to be shared between
the host and the privileged container. Is this use case even being
considered?

I don't understand what you mean by a mount namespace being shared by
the host and the privileged container. They are distinct mount
namespaces. They aren't shared. If you find the device and inode
numbers of the same path in different mount namespaces the device number
at minimum will be different. If you change one file, the other is not
affected.

Post by Rakesh
Thanks,
Rakesh
________________________________
Sent: Sunday, March 4, 2018 11:14 PM
Subject: Re: audit watch rules and docker containers

Post by Rakesh
Hello Auditd'ers,

Hi Rakesh,
(I see, with difficulty, that your output is well-formatted in the HTML
attachment, but that isn't useful. Please shut off HTML message
formatting and ensure that it looks right in plain text. Also, please
use "ls -l" so it sorts in a meaningful order for comparison.)

Post by Rakesh
I am running a privileged container with pid, net, uts space shared with the host. The need is to be able to set file watch rules from the container say
-k /etc -p rw -k containter_rule
and then look for read/write access to files/directories in /var/log/audit/*.
What I am finding is there are no watch events being logged
If I set the same audit watch rule from the host (and not being in the privileged container) I am able to get audit events
Using nsenter to switch namespace (nsenter -t 1 auditctl -k /etc -p rw -k containter_rule) does not help either
I suspect the mnt namespace is different which is causing this oddity in behavior
looking at container process namespace -
looking at init process namespace -

After decoding your jumbled mess of output due to HTML and ls options
choices, the mount namespaces are different, which would completely
explain the problem.

Post by Rakesh
Can someone please suggest with some thoughts on how to make this work.

The pending container support mentioned by Steve is not yet complete and
some more of the coming changes may help with your issue, but start by
understanding that you are examining different filesystems with your
rules above.

Post by Rakesh
Thanks,Rakesh

- RGB

- RGB

--
Richard Guy Briggs <***@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635