Near Term Audit Road Map

[Steve Grubb]

Hi,

With the proposals sent to the list, I wanted to talk about how this might
play out code-wise. With regard to the current code base, I am working on a
1.8 release. This would represent finishing the remote logging app and
nothing more. The 1.8 series would become just an update series just like the
1.0.x series did.

In parallel with finishing remote logging, I would release a 2.0 version.
Patches applied to 1.8 would also be applied to 2.0. A 2.1 release would
signify the completion of remote logging that branch. I would recommend this
branch for all distributions pulling new code in.

The 2.0 branch will also have a couple more changes. I want to split up the
audit source code a little bit. I want to drop the system-config-audit code
and let it become standalone package updated and distributed separately.

I also want to drop all audispd-plugins in the 2.0 branch and have them
released separately. They cause unnecessary build dependencies for the audit
package.

During the work for a 2.2 release, I would also like to pull the audispd
program inside auditd. In the past, I tried to keep auditd lean and single
purpose, but with adding remote logging and kerberos support, we already have
something that is hard to analyze. So, to improve performance and decrease
system load, the audit daemon will also do event dispatching.

Would this proposal impact anyone in a Bad Way?

Thanks,
-Steve

[LC Bruzenak]

Post by Steve Grubb
During the work for a 2.2 release, I would also like to pull the
audispd program inside auditd. In the past, I tried to keep auditd
lean and single purpose, but with adding remote logging and kerberos
support, we already have something that is hard to analyze. So, to
improve performance and decrease system load, the audit daemon will
also do event dispatching.
Would this proposal impact anyone in a Bad Way?

Steve,

Couple questions:
- Right now the audispd remote/prelude plugins have connection-related
concerns. For example, with the remote plugin there are timeouts,
retries, etc. and parameters to tweak that. With the prelude plugin, it
must have been registered with the running prelude-manager correctly or
it dies. Do you see that pulling in the dispatching will cause more
auditd complexity or is that not a problem? I thought that (one reason)
audispd was separate was to allow it to deal with the vagaries of
endpoint and delivery issues while the auditd kept doing its thing.

- In theory, if everything is still doing roughly the same amount of
work, I'd think that moving the functionality would not necessarily
decrease the system load. I'm sure you have thoughts on how this would
be realized and at some point I'd be interested to hear those.

- What do you see as the initial target platform for the 2.0 series in
Fedora? In RH I assume it would be the next RH release thereafter?

Overall, though, I'd vote that nothing you propose would harm my
efforts, since I think I'd be stuck with the 1.8 series for a while
anyway.

Thx,
LCB.

[LC Bruzenak]

Post by LC Bruzenak

...

audispd was separate was to allow it to deal with the vagaries of
endpoint and delivery issues while the auditd kept doing its thing.

Maybe the dispatchers themselves are still separate.

LCB.

[Steve Grubb]

Post by Steve Grubb
During the work for a 2.2 release, I would also like to pull the
audispd program inside auditd. In the past, I tried to keep auditd
lean and single purpose, but with adding remote logging and kerberos
support, we already have something that is hard to analyze. So, to
improve performance and decrease system load, the audit daemon will
also do event dispatching.

Do you see that pulling in the dispatching will cause more auditd complexity
or is that not a problem?

The current chain is:

kernel->auditd->audispd->audisp-prelude

What it would become is

kernel->auditd->audisp-prelude

I thought that (one reason) audispd was separate was to allow it to deal
with the vagaries of endpoint and delivery issues while the auditd kept
doing its thing.

Sort of. It was mostly to do the demultiplexing, but in reality, people windup
getting queue overflows between auditd and audispd.

- In theory, if everything is still doing roughly the same amount of
work, I'd think that moving the functionality would not necessarily
decrease the system load.

The problem is auditd gets backed up trying to send to audispd. Audispd rarely
gets backed up and when it does you can increase the queue. But you can't do
this between the audit daemon and dispatcher. Then auditd backs up the kernel
queue.

- What do you see as the initial target platform for the 2.0 series in
Fedora?

Either 11 or 12 depending on when I can get it done and pushed through.

In RH I assume it would be the next RH release thereafter?

Yes, it would aimed at RHEL6, with the 1.8 series becoming RHEL5 only.

-Steve

[LC Bruzenak]

I thought that we have :

(from another machine)
audisp-remote
|
v (to collector)
kernel->auditd->audispd->audisp-prelude

and that I could pick off the prelude-bound events on the aggregated
data, but I don't get the events into the prelude DB.

For example, I see the client logins in the collector's log, so the
aggregation appears to be working.
Local logins on the collector machine do get sent to prelude, so the
audisp-prelude plugin is working.

However, logins on the remote machine which are sent to the collector
log do not make it into the prelude DB (at least prewikka doesn't show
them). I have no prewikka filters and I have the prewikka viewer set to
"1 day".

Any ideas? Using 1.7.12 audit rpms.

Here is a sample of "ausearch -ts today -i -m USER_LOGIN" on the
collector:
...
node=v157 type=USER_LOGIN msg=audit(03/24/2009 10:44:27.533:548759) :
user pid=11353 uid=root auid=root ses=328
subj=system_u:system_r:sshd_t:s0-s15:c0.c1023 msg='uid=root
exe=/usr/sbin/sshd (hostname=homeserver, addr=192.168.31.40,
terminal=/dev/pts/0 res=success)'
----
node=audit type=USER_LOGIN msg=audit(03/24/2009 11:11:37.882:1412) :
user pid=3103 uid=root auid=root ses=54
subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 msg='uid=root
exe=/usr/sbin/sshd (hostname=192.168.31.40, addr=192.168.31.40,
terminal=/dev/pts/3 res=success)'

On the prewikka screen I only see the second event.

Thx,
LCB.

[Sebastien Tricaud]

Post by LC Bruzenak
However, logins on the remote machine which are sent to the collector
log do not make it into the prelude DB (at least prewikka doesn't show
them). I have no prewikka filters and I have the prewikka viewer set to
"1 day".

Hi,

can you see events coming to prelude-manager when running:
prelude-manager --debug

?

Thanks,
Sebastien.

[LC Bruzenak]

Post by Sebastien Tricaud

Post by LC Bruzenak
However, logins on the remote machine which are sent to the collector
log do not make it into the prelude DB (at least prewikka doesn't show
them). I have no prewikka filters and I have the prewikka viewer set to
"1 day".

Hi,
prelude-manager --debug
?
Thanks,
Sebastien.

Sebastien,

Thanks for the reply. I do not see it getting into the prelude-manager.
I also did a local login so I could see one which worked and the alert
came through in that case.

LCB.

[Steve Grubb]

Post by LC Bruzenak
On the prewikka screen I only see the second event.

prelude is its own protocol and picks out certain data from its config files and
puts in its packets. The intended use is each machine sends its prelude alerts
to a common prelude manager. Each audit event is sent to its aggregator. The
two systems diverge at audispd.

kernel->auditd->audispd-+->audisp-prelude->prelude-manager
+->audisp-remote->auditd

-Steve

[LC Bruzenak]

Post by Steve Grubb

Post by LC Bruzenak
On the prewikka screen I only see the second event.

prelude is its own protocol and picks out certain data from its config files and
puts in its packets. The intended use is each machine sends its prelude alerts

not MY intended use...
:)

Post by Steve Grubb
to a common prelude manager. Each audit event is sent to its aggregator. The
two systems diverge at audispd.
kernel->auditd->audispd-+->audisp-prelude->prelude-manager
+->audisp-remote->auditd
-Steve

Steve; thanks.

I may not follow. Does the above preclude what I'm asking?
Asked another way, what stops the aggregated audit events from creating
a prelude event?

Thx,
LCB.

[Steve Grubb]

Post by LC Bruzenak
Asked another way, what stops the aggregated audit events from creating
a prelude event?

Prelude grabs things from its own config files to fill in certain fields. This
means that if run from an aggregator, it will use the same values for all
events. This affects the host names that show up in heartbeats and other
events.

The two are meant to be separate but complimentary systems.

-Steve

[Steve Grubb]

Post by LC Bruzenak
On the prewikka screen I only see the second event.

prelude is its own protocol and picks out certain data from its config files and
puts in its packets. The intended use is each machine sends its prelude alerts
to a common prelude manager. Each audit event is sent to its aggregator. The
two systems diverge at audispd.

kernel->auditd->audispd-+->audisp-prelude->prelude-manager
+->audisp-remote->auditd

-Steve

[Matthew Booth]

Post by Steve Grubb
Hi,
With the proposals sent to the list, I wanted to talk about how this might
play out code-wise. With regard to the current code base, I am working on a
1.8 release. This would represent finishing the remote logging app and
nothing more. The 1.8 series would become just an update series just like the
1.0.x series did.
In parallel with finishing remote logging, I would release a 2.0 version.
Patches applied to 1.8 would also be applied to 2.0. A 2.1 release would
signify the completion of remote logging that branch. I would recommend this
branch for all distributions pulling new code in.
The 2.0 branch will also have a couple more changes. I want to split up the
audit source code a little bit. I want to drop the system-config-audit code
and let it become standalone package updated and distributed separately.
I also want to drop all audispd-plugins in the 2.0 branch and have them
released separately. They cause unnecessary build dependencies for the audit
package.
During the work for a 2.2 release, I would also like to pull the audispd
program inside auditd. In the past, I tried to keep auditd lean and single
purpose, but with adding remote logging and kerberos support, we already have
something that is hard to analyze. So, to improve performance and decrease
system load, the audit daemon will also do event dispatching.
Would this proposal impact anyone in a Bad Way?

On the contrary. My austream tool was born because:

* Ensuring a dispatcher doesn't generate audit events is fragile
* The additional task switching and memory copying becomes onerous under
load

Additionally, auditd is clearly geared up for writing to disk: certainly
in RHEL 4, switching off all disk related activity is a whole lot of
typing to tell it not to do anything :)

Solaris's BSM implements custom behaviour with loadable modules. If our
auditd did that, hopefully I could deprecate austream. The dispatcher
architecture doesn't lend itself to sustained high volume.

Matt