On Wed, 11 Apr 2001, Peter Jeremy wrote:
> > It has been
> > suggested to me that these would make a great addition to a new
> > regression/ CVS sub-tree, that they should be included under
> > appropriate existing parts of the tree reflecting what they test,
> > and that they would be best in the form of a port.
>
> I'd prefer to see any test code in the tree near the code being tested
> - this (marginally) improves the chances of tests being updated to
> cover new functionality. A number of parts of the system already
> include stand-alone self-checks which could probably be invoked as
> part of an overall regression test. Pushing the tests into a
> separate port makes it more difficult for developers to use the
> tests and would appear to make it harder to update the tests.
>
> Any generic test skeleton files probably belong in a "regression"
> tree under /usr/src.
Where do you think that kernel regression tests should be placed? Often,
these tests will be initiated and managed from userland, so they are
probably inappropriate to drop in sys/?
> Since you are talking about changing the security-related behaviour of
> the system, I'd prefer it to require an "options REGRESSION" (and maybe
> then set a sysctl), rather than be a KLD. I'm not at all keen on the
> idea of having it stashed away as a separately maintained port.
One of my primary motivations for looking at importing the features into
the base kernel was to avoid both the non-use, bitrot and synchronization
problems currently inherent to ports. Ports with kld's suffer breakage in
a number of ways, due to lack of synchronization with the base tree,
difficulty in maintaining up-to-date kld builds when the system is a
moving target, etc. (For example, I rebuild my workstation almost daily
on -CURRENT -- and I have to rebuild my vmware kld's about as frequently
or risk panics). For many consumers sitting on a release, this may be
less of a problem, but for a development tool on the -CURRENT branch, it's
a serious problem.
Does this mean we need a <sys/regression.h> which provides constants and
prototypes for regression-specific interfaces which are enabled using
options REGRESSION? (This is how I have it implemented in my local source
tree, but I'm interested in whether this seems like a good idea to anyone
else :-).
> > Is the idea of helper functions for regression testing simply
> > philosophically wrong, or does it serve a useful function
>
> I think it's a valid part of white-box testing. In any piece of
> software, there are going to be code paths that are very difficult or
> impossible to exercise using the `normal' interfaces. Providing
> interfaces that are intended solely for testing the correctness of the
> code seems perfectly reasonable. This is no different to the JTAG ports
> on virtually every modern LSI chip.
Sounds good to me -- I think we agree on almost all points.
Robert N M Watson FreeBSD Core Team, TrustedBSD Project
robert@fledge.watson.org NAI Labs, Safeport Network Services
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message