Kernel Traffic #335 by Zack Brown
[email protected] Sun, 27 Nov 2005 20:49:45 -0800
| Newsgroups | gmane.linux.kernel.kernel-traffic |
|---|---|
| Message-ID | <[email protected]> |
Kernel Traffic #335 For 27 Nov
By Zack Brown
It's always worked for me, but hey, maybe I'm just lucky.
--Linus Torvalds
Table Of Contents
* Standard Format
* Text Format
* XML Source
* Mailing List Stats For This Week
* Threads Covered
1. 29 Oct - 7 Nov (78 Describing And Debating The Development
posts) Process
2. 31 Oct - 4 Nov (15 Linux 2.4.32-rc2 Released
posts)
3. 2 Nov - 4 Nov (7 Sharp Zaurus-5500 Testers Wanted
posts)
4. 2 Nov - 7 Nov (42 New eCryptFS Filesystem
posts)
5. 3 Nov - 5 Nov (9 git Binary Safe
posts)
6. 3 Nov - 5 Nov (5 git 0.99.9c Released; Querying The git
posts) Network
7. 4 Nov (3 Fixing Incorrect Authors And Committers In
posts) git Repositories
8. 5 Nov - 9 Nov (18 Ubuntu Kernel Maintained In git
posts)
9. 6 Nov - 7 Nov (5 git 0.99.9e Released
posts)
10. 7 Nov (3 New Trivial Patch Monkey Maintainer
posts)
11. 7 Nov (1 post) Another Attempt To Remove DevFS
12. 8 Nov (1 post) Secure Digital Host Controller Support
13. 4 Nov - 9 Nov (19 Linux 2.6.14.1 Released
posts)
14. 9 Nov - 10 Nov (6 Linux 2.4.32-rc3 Released
posts)
Mailing List Stats For This Week
We looked at 2243 posts in 14MB. See the Full Statistics.
There were 662 different contributors. 246 posted more than once. The average
length of each message was 106 lines.
+-----------------------------------------------------------------------------+
| The top posters of the week | The top subjects of the week were: |
| were: | |
|--------------------------------+--------------------------------------------|
| |78 posts in 364KB for "new (now current |
|88 posts in 386KB by |development process)" |
|[email protected] |41 posts in 172KB for "3d video card |
|74 posts in 515KB by russell |recommendations" |
|king |38 posts in 178KB for "best way to handle |
|73 posts in 560KB by adrian bunk|leds" |
|68 posts in 384KB by |34 posts in 160KB for "[patch]: clean up of |
|[email protected] |__alloc_pages" |
|51 posts in 492KB by greg kh |33 posts in 159KB for "ntp broken with |
| |2.6.14" |
+-----------------------------------------------------------------------------+
These stats generated by mboxstats version 2.8
1. Describing And Debating The Development Process
29 Oct - 7 Nov (78 posts) Archive Link: "New (now current development
process)"
Topics: Bug Tracking, Disks: SCSI, PCI
People: Paolo Ciarrocchi, Jesper Juhl, Tony Luck, Russell King, Linus Torvalds
, Christopher Friesen, Willy Tarreau, Andi Kleen, Alexander Viro, Andrew Morton
, Martin J. Bligh, Zwane Mwaikambo, Roland Dreier, Tim Bird, Roman Zippel, Eric
Sandall, Theodore Y. Ts'o, Krzysztof Halasa
Paolo Ciarrocchi said:
I would like to write a short article about the new development process
discussed during last Linux Kernel Developers' Summit for my local LUG.
Since I'm not able to find an accurate report of what has been discussed during
that meeting I try to summariza what is my understanding of the current
process:
The are two kind of releases, 2.6.x kerneles and 2.6.x.y
2.6.x.y are maintained by the "stable" team (stable at kernel dot org), are
released amost every week.
Here is the latest ( I hope ) announce I found in the archive including an
explanation of how stable development works:
Rules on what kind of patches are accepted, and what ones are not, into the
"-stable" tree:
* It must be obviously correct and tested.
* It can not bigger than 100 lines, with context.
* It must fix only one thing.
* It must fix a real bug that bothers people (not a, "This could be a
problem..." type thing.)
* It must fix a problem that causes a build error (but not for things marked
CONFIG_BROKEN), an oops, a hang, data corruption, a real security issue, or
some "oh, that's not good" issue. In short, something critical.
* No "theoretical race condition" issues, unless an explanation of how the
race can be exploited.
* It can not contain any "trivial" fixes in it (spelling changes, whitespace
cleanups, etc.)
* It must be accepted by the relevant subsystem maintainer.
* It must follow Documentation/SubmittingPatches rules.
Procedure for submitting patches to the -stable tree:
* Send the patch, after verifying that it follows the above rules, to
[email protected].
* The sender will receive an ack when the patch has been accepted into the
queue, or a nak if the patch is rejected. This response might take a few
days, according to the developer's schedules.
* If accepted, the patch will be added to the -stable queue, for review by
other developers.
* Security patches should not be sent to this alias, but instead to the
documented [email protected].
Review cycle:
* When the -stable maintainers decide for a review cycle, the patches will be
sent to the review committee, and the maintainer of the affected area of
the patch (unless the submitter is the maintainer of the area) and CC: to
the linux-kernel mailing list.
* The review committee has 48 hours in which to ack or nak the patch.
* If the patch is rejected by a member of the committee, or linux-kernel
members object to the patch by bringing up issues that the maintainer and
members did not realize, the patch will be dropped from the queue.
* At the end of the review cycle, the acked patches will be added to the
latest -stable release, and a new -stable release will happen.
* Security patches will be accepted into the -stable tree directly from the
security kernel team, and not go through the normal review cycle. Contact
the kernel security team for more details on this procedure.
Review committe:
* This will be made up of a number of kernel developers who have volunteered
for this task, and a few that haven't.
2.6.x kernel are maintained by Linus Torvalds and Andrew Morton (both are from
OSDL) and the development is as follow:
* As soon a new kernel is released a two weeks windows is open, during this
period of time maintainers can submit big diffs to Linus (usually the
patched sitted in -mm kernels for a few weeks). Preferred way to submit big
changes is using GIT ( more information about GIT at http://git.or.cz/ and
http://www.kernel.org/pub/software/scm/git/docs/tutorial.html).
* After two weeks a -rc1 kernel is released and now is possible to push only
patches that do not include new functionalities)
* Aftwer two weeks -rc2 is released
* Process continues until the kernel is considered "ready", the process
should last three months ( 4 kernels per yeard should be released).
I'm sure I'm missing lot of details so I would really appreciate if you could
support me in writing this article that hopefully will be usefull outside my
LUG as well :-)
Jesper Juhl replied:
I recently wrote a document on the different kernel trees and how to apply
patches for them. In that document I also give a description of the various
trees.
Perhaps that document will be useful to you. You can find it in a recent kernel
source tree as Documentation/applying-patches.txt or you can read it online via
lrx at this URL: http://sosdg.org/~coywolf/lxr/source/Documentation/
applying-patches.txt
Good luck on your article. Once you are done with it I'd love to read it and I
guess a lot of other people would too, so please supply an URL to it at that
time :-)
Elsewhere, Tony Luck replied to Paolo's initial post. Regarging the description
of the -rc1 timeline, Tony said, "Initially Linus said that he would only
accept e-mail patches after rc1 ... but common sense prevailed and he later
clarified to say that git merges could still be used, but only to include bug
fixes." He added, "Also note that a whole new driver (or filesystem) might be
accepted after -rc1 because there is no risk of causing regressions with such a
change." Tony also pointed out that the only release candidate with an official
schedule was -rc1. Subsequent candidates happened when they happened. Tony also
addressed Paolo's description of the three month schedule between official
kernel releases. He said, "the goal was to make a release in around 8 weeks
(which would be closer to six per year). But you have the important part, which
is that Linus doesn't make the release until it is "ready". 2.6.13 was released
on August 28th, and 2.6.14 on October 27th ... so we appear to have hit the
goal this time through." Russell King replied:
However, three months is _far_ too long. Look at what is happening post-2.6.14?
There's loads of really huge changes going in which were backed up over the
last two months.
Continuing like this will just push the release of each kernel further and
further out as there's more stuff merged during the mayhem than can possibly be
properly reviewed and tested.
Shorter release cycles means that there's less mayhem, which in turn means more
time to review.
And Linus Torvalds said:
Yes. Three months would be too much. I considered even 2.6.14 to be delayed by
at least one week. So 8 weeks is certainly better than it used to be, but I
think 6 weeks should be the goal-post.
But to hit that, we'd might need to shrink the "merge window" from two weeks to
just one, otherwise there's not enough time to calm down. With most of the real
development happening during the previous calm-down stage, that might not be
impossible: we'd only have one week for merges, but that isn't the week when
development actually happens, so who cares?
Everything said, I think 2.6.13->14 worked well enough, even if it's hard to
say how well a process works after one release. Considering that 2.6.13 had the
painful PCI changes (you may not have noticed too much, since they were x86
only) and there were some potentially painful SCSI changes in the .14 early
merges, so it's not like 13->14 was an "easy" release - so the process
certainly _seems_ to be workable.
So I'm planning on continuing with it unchanged for now. Two-week merge window
until -rc1, and then another -rc kernel roughly every week until release. With
the goal being 6 weeks, and 8 weeks being ok.
I don't think anybody has been really unhappy with this approach? Hmm?
There were a couple of subthreads emanating from this one. In one of them,
several folks began discussing the relative merits of having an official kernel
release be unchanged from the most recent -rc release. Eric Sandall argued that
the whole point of an -rc release was to create a "release candidate", i.e.
something that would become the official release if no further problems arose.
Krzysztof Halasa argued that it was okay for the official release to be
different from the most recent -rc release, so long as no new bugs were
introduced. Christopher Friesen pointed out that it was difficult to know that
no new bugs had been introduced. Christopher said, "The safe bet is to simply
rename the final -rc with no further changes." But Linus replied:
That's not actually a very safe bet at all.
Why? Because only a small percentage of people actually test -rc kernels in the
first place.
So if you release the last -rc as the standard kernel, you (a) get no real
coverage advantage anyway and (b) you just wasted a week in order to try to get
some coverage that you aren't getting.
The current kernel development model is to merge stuff early, which hopefully
motivates the people who _do_ test -rc kernels to actually test -rc1, since
they know that that's the one that has _all_ the really relevant goodies.
And most of those that do test -rc1 will never see any problems at all. Those
that do, are now more likely to test the rest of the -rcs, hopefully until
their problem is resolved. And those that don't test -rc releases (because they
simply don't upgrade very often) will _never_ test an -rc release, whether it's
the first one or the last one.
So what we have to fight this problem is the stable kernel series. We release
the final 2.6.x kernel with as much testing as we can, but it's just an
undeniable fact that a lot of people will try that kernel only after the
release, and often it might be weeks after the release. Doing -rc kernels
didn't do anything for those cases.
But when they find a problem (or somebody who _did_ test an -rc kernel, but
didn't notice a problem until much later), we try to have a process to get
those issues fixed.
So repeat after me: "Most people never test -rc kernels".
He continued in reply to himself:
Btw, the ones that _do_ test -rc kernels usually don't test all of them. The
current model is set up in a way where there is _one_ special -rc kernel that
we should try to get people to test: the first one.
That hopefully encourages people to try an -rc kernel who might otherwise
decide that there's too many -rc kernels to bother with. If they know that all
of the real development happened before -rc1, they also are thus aware that it
doesn't really matter which -rc kernel they test, so just testing _one_ is very
good indeed.
The first -rc kernel is also special in another way: it's the one we "wait"
for. It's the one that happens after two weeks, and has a deadline. The others
happen more frequently, and are really objectively less important than the
first one.
(In contrast, some other projects try to make the _last_ -rc be the important
one. That's totally the wrong way around, because if there are more people
testing the last one, the testing happens at _exactly_ the wrong point in time
from a "let's fix the problems" standpoint)
So the call to people who can be bothered to test at all: if you only test one
-rc kernel, please test the first one. That way we get a heads-up on problems
earlier.
(And if you like testing -rc kernels, please test all of them. Or even the
nightly snapshots. Or track the git tree several times a day. The more, the
merrier, but if you only want to boot one kernel a month, make it be the -rc1
kernel).
Willy Tarreau pointed out that at least for some developers, the situation was
different from what Linus described. He said:
I *try* to test *ALL* versions which are announced as the most likely future
-final. When Marcelo tells us that -rc2 will be -final if nobody complains, I
really test it. I build it on several archs and try to catch stupid bugs
because I hate stupid bugs in final releases, they pollute bug reports (worst
ones being build errors).
However, when he announces -preX (X>1) or when you annonce any -rc which is not
likely to become -final, I just check the changelog, and build it if I both see
something which applies to my setup, and I have nothing else to do.
Elsewhere, Andi Kleen also replied to Linus's request for feedback on the
current development approach. He said, "The long freeze periods were nothing
much happens are painful. It would be better to have some more overlap of
merging and stabilizing (stable does that already kind of, but not enough)" .
Theodore Y. Ts'o replied that Andrew Morton accepted patches into the -mm tree
during the freeze periods, and so it couldn't really be called a period where
"nothing much happens". But Andi replied, "The problem is that -mm* contains
typically so many more or less broken changes that any extensive work on there
is futile because you never know whose bugs you're debugging (and if the patch
that is broken will even make it anywhere). In short mainline is frozen too
long and -mm* is too unstable." And Alexander Viro added, "Besides, -mm is
changing so fscking fast that it doesn't build on a lot of configs most of the
time. And trying to keep track of it and at least deal with build breakage at
real time is, IME, hopeless." He added later that he routinely tried building
-mm releases for a wide range of configurations, and found large numbers of
problems. Andrew Morton pointed out that "Every -mm release is built with
allmodconfig on x86 and on x86_64. It's also cross-compiled on fat configs for
alpha, ppc32, ppc64, sparc64, arm and ia64. It's booted on x86, x86_64, ppc64
and ia64." He added, "I usually do allnoconfig and allyesconfig, too."
Alexander said he tested with a much broader array of configurations. He listed
some off, and said, "_IF_ somebody wants to do that for -mm, yell and you are
more than welcome to all infrastructure, except for the cycles on build box I'm
using. Incidentally, it is a box at work - my energy bill is high enough as it
is, without adding an 8-way 3GHz iamd64 to it..." Martin J. Bligh replied,
"I'll do that if you want. I have a big lab full of largish boxes with serious
aircon, and IBM can afford the power bill. I'm assuming this is a farm of
cross-compilers that'll run on x86 (or x86_64)?" But the thread ended or went
private at that point.
Elsewhere, also in reply to Andi's initial criticism of the long freeze
periods, Russell King agreed strongly, saying, "I find the long freeze periods
painful and very very very boring, to the point of looking for other stuff to
do (such as cleaning up bits of the kernel and queuing mega-patches for the
next round of merging.)" Andrew replied:
The freezes are for fixing bugs, especially recent regressions. There's no
shortage of them, you know.
I you can think of a better way to get kernel developers off their butts and
actually fixing bugs, I'm all ears.
He added later:
a) you're sitting around feeling very very very bored while
b) the kernel is in long freeze due to the lack of kernel developer attention
to known bugs
The solution seems fairly obvious to me?
Russell said, "That's fine if you have the hardware to be able to debug these
issues." And Andrew replied:
Most driver bugs cannot be reproduced by the developer. Almost by definition:
if the patch had caused problems on the developer's machine, he wouldn't have
shipped it.
This is why we have this wonderful group of long-suffering people who download
and test development kernels for us, and who take the time to report defects.
Yes, it's painful to get into a long-range debugging session, sending debug
patches, twelve-hour turnaround, etc. But what alternative have we?
Close by, Andi said:
The problem is that you usually cannot do proper bug fixing because the release
might be just around the corner, so you typically chose the ugly workaround or
revert, or just reject changes for bugs that a are too risky or the impact too
low because there is not enough time to properly test anymore.
It might work better if we were told when the releases would actually happen
and you don't need to fear that this not quite tested everywhere bugfix you're
about to submit might make it into the gold kernel, breaking the world for some
subset of users.
Andrew replied:
Nobody knows when a kernel will be released, because it's released according to
perceived bug status, not according to a preconceived timeline.
I just don't buy what you're saying. Unless the kernel is at -rc3 or -rc4, we
*know* the release is weeks away - it's always been thus.
He added, "There is absolutely nothing preventing people from working on bugs
at any time! It's not as if a bugfix can ever come too early."
Close by, Russell backed Andi up, saying:
Indeed - a prime example is the bootmem initialisation problem. Had I known on
1st October that the final release wouldn't have been for a long time, I'd have
applied that patch and you wouldn't have had that problem with ARM relying on
the bootmem initialisation order clashing with your ia64 (or x86_64) swiotlb
fix.
But stupid rmk - again I hasten to add - was suckered into this "-rc is frozen"
idea again and decided it wasn't appropriate to submit it. In hind-sight, there
would have been plenty of time to sort out any issues.
Hell, we held up 2.6.14 for swiotlb _anyway_.
Andrew said:
I'd say that was an *exception*, not a "prime example".
I've filed away 322 unresolved bug reports here. The great majority are busted
drivers on random hardware dating back as far as 2.6.11. Many of them are
regressions.
There is nothing stopping anyone from working with the originators to get these
things fixed up at any time.
Why is it necessary for me to chase maintainers to get their bugs fixed?
Why are maintainers working on new features when they have unresolved bugs?
Why is it so often me who has to do the followup for un-responded-to bug
reports against subsystems which I don't know anything about?
(Those are rhetorical questions, btw).
Andi said:
Because zero bugs is just unrealistic and they would never get anything done if
that was the requirement?
(especially considering that a lot of the bugs at least on x86-64 are hardware/
firmware bugs of some sort, so often it is not really a linux bug but just a
missing ha^w^wwork^w^w^w^wfix for something else)
I agree regressions are a problem and need to be addressed, but handling all
non regressions on a non trivial platforms is just impossible IMHO...
Perhaps it would be nice to have better bug classification: e.g. regression/new
hardware/reported by more than one person etc. I think with some prioritization
like that it would be much easier to keep the bugs under control.
Sometimes bugs are less important than others. e.g. on x86-64 the bootmem issue
was obscure at best, affecting only a very small part of the user base. Sure
the few people hit by it will be annoyed, but trying to make everyone happy is
impossible so one has to try to just make the majority of users happy. So it
was imho ok to just revert the patch to fix ARM again and not breaking IA64 (I
cannot assess how many users were on ARM/IA64 affected) for the next release
and try to sort it out later.
Regressions are important, everything else has to be prioritized based on the
impact on the user base (and this doesn't necessarily mean the most noisy part
of the userbase)
Perhaps some people could volunteer to set some flags in bugzilla for obvious
things, like regression or new hardware or missing basic information or for
really old kernel and no report for a new one and that could be used to filter
the queries better. Should be an relatively easy task.
Zwane Mwaikambo remarked, "I don't think we need any special flags, we just
need more people paying attention to it. It doesn't take that much time to go
through pending bugs and trying to identify real ones, but the problem is that
we need knowledgeable people to do this." And Andrew added, "I don't believe
that what we're seeing is some prioritisation process. The problem is that some
bugs are *harder* than others, and people are just ducking things which cannot
be locally reproduced."
At this point, the discussion veered sharply off, when Roman Zippel pointed out
that the compiled kernels were rapidly growing in size. He listed a bunch of
kernels, starting with 2.2.18 at 495K, and going up to 2.6.14 at over three
times that size.
Andrew asked, "Are you sure these kernels are feature-equivalent?" And Linus
replied:
They may not be feature-equivalent in reality, but it's hard to generate
something that has the features (or lack there-of) of old kernels these days.
Which is problematic.
But some of it is likely also compilers. gcc does insane padding in many cases
these days.
And a lot of it is us just being bloated. Argh.
A few posts later, he added:
I really don't see the point of shaving less than a kB with ugly calling
convention magic, when switching to -Os can save us much more, and when the
networking code is several hundred kB.
If we start doing size optimizations, we need to think big.
Roland Dreier said, "it would be great to find ways to make big improvements.
But I think the most realistic way to shrink the kernel is the same way it
grows in the first place -- one small piece at a time." Linus replied:
No, I think that's a lost cause.
It doesn't grow by 700 bytes once in a while. It grows by much more, and much
more often. And we can't fight it that way, that's just not going to work.
Maybe have something that tracks individual object file sizes and shames people
into not growing them..
Tim Bird replied:
As mentioned at the kernel summit, we're working on it at the CE Linux Forum.
These things take time to set up, but we have code already for something that
tests the size impact of individual kernel configs, and we're working on a test
to track individual function sizes for each new kernel (using Andi Kleen's
bloat-o-meter).
We're still in process of setting up the test lab, but we have a number of
hardware boards already, and we're hoping to be publishing size regression data
for the kernel on a regular basis, starting in about April of next year.
Andi asked, "Any particular reason it's taking so long? Would it perhaps be
possible to do a subset (e.g. only a single architecture and less configs)
sooner? If the problem is writing the necessary scripts, I'm sure some people
(e.g. M. Bligh or Al Viro) have already written some that do at least the
automated building and would be happy to share them." And Tim said:
The main reason it's taking so long is that it's a background task for everyone
working on it. (Basically, me and one guy at NEC. ;-) We've gotten sidetracked
with lab setup issues (signing a lease for office space, processing paperwork
for board donations, etc. etc.) But I can run some tests now on a few different
platforms just on my desk.
I had hoped to have at least some preliminary results by the end of November,
with a bigger call for help inside the forum to finish up something big and
professional-looking in April. However, maybe I can cobble together some
things, and let people (both in and out of the forum) look at the preliminary
results sooner.
I'll see what I can do by the end of November.
2. Linux 2.4.32-rc2 Released
31 Oct - 4 Nov (15 posts) Archive Link: "Linux 2.4.32-rc2"
Topics: Networking, SMP, USB
People: Marcelo Tosatti, Dan Aloni, Pete Zaitcev, Julian Anastasov, Ralf
Baechle, Nick Piggin, Alexey Kuznetsov, Andrew Morton, Willy Tarreau
Marcelo Tosatti announced Linux 2.4.32-rc2, saying:
Here goes the second release candidate for v2.4.32.
The most significant changes are v2.6 backports of IPv4/IPv6 bugfixes, and a
USB OHCI regression introduced during v2.4.28 which could lead to crashes on
SMP kernels.
He posted the changelog:
Aleksey Gorelov:
asus vt8235 router buggy bios workaround
Alexey Kuznetsov:
[TCP]: Don't over-clamp window in tcp_clamp_window()
Andrew Morton:
loadkeys requires root priviledges
Dan Aloni:
fix memory leak in sd_mod.o
Denis Lukianov:
[MCAST]: Fix MCAST_EXCLUDE line dupes
Herbert Xu:
Clear stale pred_flags when snd_wnd change
Horms:
[IPVS]: Add netdev and me as maintainer contacts
Fix infinite loop in udp_v6_get_port()
Julian Anastasov:
[IPVS]: ip_vs_ftp breaks connections using persistence
[IPVS]: really invalidate persistent templates
Marcelo Tosatti:
Change VERSION to 2.4.32-rc2
Marcus Sundberg:
[NETFILTER]: this patch fixes a compilation issue with gcc 3.4.3.
Nick Piggin:
possible memory ordering bug in page reclaim
Pete Zaitcev:
usb: regression in usb-ohci
Ralf Baechle:
AX.25: signed char bug
Willy Tarreau:
Fix jiffies overflow in delay.h
Later, he added:
There is nothing else pending for v2.4.32 on my part.
Will wait a couple of days and tag it as v2.4.32.
3. Sharp Zaurus-5500 Testers Wanted
2 Nov - 4 Nov (7 posts) Archive Link: "sharp zaurus-5500: looking for testers
"
Topics: Touchscreen
People: Pavel Machek, Carlos Martin
Pavel Machek said:
Is there someone out there, with sharp zaurus sl-5500, willing to test kernels?
There's a linux-z tree on kernel.org, which I try to more or less keep in sync
with mainline, that is slowly starting to get usable. It could use some
testing.
Main drawback is that battery charging is not yet done; touchscreen is there
but I did not have chance to test it with proper userspace filtering.
Keenan Pepper leapt through his monitor to hug Pavel, and said he'd begin
testing as soon as he could. Carlos Martin also grabbed Pavel's patches, but
reported, "your git tree (repository?) in kernel.org is a bit broken. The git
web interface gives me 403 error when I try to see a diff in your zaurus.git
tree, and there's stuff that appears to be missing (history and commits)."
Pavel worked on fixing this, and a couple days later reported that the
repository should be working again. Carlos confirmed this, and said he'd report
on his tests as soon as they were finished.
4. New eCryptFS Filesystem
2 Nov - 7 Nov (42 posts) Archive Link: "[PATCH 0/12: eCryptfs] eCryptfs
version 0.1"
Topics: Ottawa Linux Symposium, Version Control
People: Phillip Hellewell, David Howells, Erez Zadok
Phillip Hellewell said:
This set of patches constitutes eCryptfs version 0.1. We are presenting it to
be reviewed and considered for inclusion into the kernel.
eCryptfs is a stackable filesystem that is based off of the Cryptfs that is
generated by the FiST stackable filesystem framework written by Erez Zadok:
http://filesystems.org/
eCryptfs stores cryptographic metadata in the headers of each file; the headers
contain OpenPGP-like packets (see RFC 2440). This allows the encrypted
underlying files to be copied between hosts, and all of the information
necessary to decrypt the files stays with the files themselves. eCryptfs aims
to make the encryption and the decryption of each individual file completely
transparent to userspace applications, so long as the recipient has the
requisite key or passphrase to access the file available.
Michael Halcrow presented eCryptfs at the 2004 and the 2005 Ottawa Linux
Symposiums; the high-level overview from this year's symposium starts on page
209 of the first half of the symposium proceedings:
http://www.linuxsymposium.org/2005/linuxsymposium_procv1.pdf
Note that this set of patches contains a considerably trimmed-down version of
eCryptfs than what was sent to the LKML earlier this year. Release 0.1 includes
mount-wide passphrase support only; this will make eCryptfs easier to analyze
and debug before the more advanced policy and public key features are merged
in.
eCryptfs performs well under a variety of tests, including FSX and Connectathon
(Basic and General functional). There is a bug that crops up on a kernel
compile. We would appreciate any insight that the VFS guru's could give us in
tracking down and fixing any extant bugs.
eCryptfs utilizes David Howells' keyring; at mount, eCryptfs version 0.1
expects an existing authentication token in the user's session keyring. The
tarball containing the code to do this is available from the eCryptfs
SourceForge site (ecryptfs-v0_1.tar.bz2):
http://sourceforge.net/projects/ecryptfs/
Future releases will have policy support, which will entail per-file passphrase
and per-file public key support. Those who are interested in looking at that
code are welcome to obtain it from the eCryptfs CVS repository on SourceForge:
cvs -d:pserver:[email protected]:/cvsroot/ecryptfs login
cvs -z3 -d:pserver:[email protected]:/cvsroot/ecryptfs co -P
ecryptfs
5. git Binary Safe
3 Nov - 5 Nov (9 posts) Archive Link: "binary safe?"
Topics: Version Control
People: Randal L. Schwartz, Linus Torvalds, Nick Hengeveld
On the git mailing list, Randal L. Schwartz said, "I'm currently about to
abandon CVS for my website management, replacing it with git." He asked if git
was binary-safe, in the sense that it could be used to house non-text data.
Linus Torvalds replied:
Yes. I don't think it's been heavily tested, but the very architecture of git
should mean that there just shouldn't be any issues with binary files outside
of the obvious ones.
The only binary file the kernel ever uses is the logo.gif thing, so it's been
"tested" in the sense that binary files exist, but there's never been any
changes to that file, so..
Nick Hengeveld also remarked, "We're now using git in production to distribute
content, of which well over 90% is binary files. Works great."
6. git 0.99.9c Released; Querying The git Network
3 Nov - 5 Nov (5 posts) Archive Link: "GIT 0.99.9c"
People: Junio C. Hamano, Jeff Garzik, Junio C Hamano, Alex Riesen
Junio C. Hamano announced git 0.99.9c, saying:
GIT 0.99.9c is found at http://kernel.org/pub/software/scm/git/ as usual. It
includes the following fixes and documentation updates since 0.99.9b:
Alex Riesen:
remove CR/LF from .gitignore
Jon Loeliger
Illustration: "Fundamental Git Index Operations"
Illustration: "Git Diff Types"
Illustration: "Commit DAG Revision Naming"
Junio C Hamano:
Do not put automatic merge message after signed-off-by line.
git-clone: do not forget to create origin branch.
Make test-date buildable again.
Do not fail on hierarchical branch names.
Ignore '\r' at the end of line in $GIT_DIR/config
Be careful when dereferencing tags (credits Pasky).
Document --since and --until options to rev-parse.
Add --no-commit to git-merge/git-pull.
Add 'ours' merge strategy.
git-merge-ours: make sure our index matches HEAD
GIT 0.99.9c
Peter Eriksen:
Clean up the SunOS Makefile rule
The slow and steady march toward 1.0 continues.
I plan to do another full sweep in the documentation directory on my next GIT
day.
On the proposed updates front, I am hoping to include Nick's http-push using
DAV in the "master" branch soon. And I would appreciate somebody who actually
uses svnimport to Ack on Yaacov's svnimport fix.
Jeff Garzik replied:
Here's an experiment I've been dying to try.
The current "tracker-less" BitTorrent (http://www.bittorrent.com/
trackerless.html) employs a distributed hash table (http://www.etse.urv.es/
~cpairot/dhts.html) called Kademlia, where the total content is spread across a
bunch of computers on the network. I kinda prefer TANGLE (http://
www.nicemice.net/amc/research/tangle/) to Kademlia.
Anyway, I was thinking that it would be a neat experiment to add simple
TANGLE-like peer-to-peer code, to enable git to query "the git network hash
table" for content.
Comments, or any pre-code-creation objections? How easy is it to add a new
storage backend to git?
To restrict unlimited uploading, I'm thinking that I'll want the system to fall
back to {www,git,rsync}.kernel.org as the original source of content. [though
the code will obviously be generic, and not hardcode *.kernel.org policy]
Junio addressed the issue of adding a new storage back-end, saying:
Almost everything is contained within sha1_file.c.
Object creation side is simple -- everybody who creates an object (e.g
update-index registering blobs, write-tree writing the toplevel and
intermediate level trees, commit-tree building a commit object, unpack-objects
exploding a pack) goes through write_sha1_file(), which checks if the object is
already available using has_sha1_file() and creates a new object in the local
.git/objects/?? directory. I am assuming that you are not planning to create
objects in a remote peer from within the git code path, and instead to have
background process that replicate them over the network to peer repositories,
so you probably do not have to touch this side.
Extending inspection and reading from existing objects for your networked
storage may be somewhat messy, but starting points are:
* has_sha1_file() takes the object name and returns true/false; if the object
is available to us in any form (be it in one of the alternate object stores
or the local repository, as individual object in an objects/??/ file or
stored in a pack). Currently it looks at packs and then checks individual
files for performance reasons (to minimize seeks and to prevent polluting
dcache with many negative hits); you would be adding another data source.
* read_sha1_file() takes the object name, and returns the uncompressed
contents of the object in addition to the type and the size.
* sha1_object_info() takes the object name and returns the type of the object
and optionally returns the size of it, without reading the contents. Some
programs call this before reading the data using read_sha1_file(), so you
might want to use this as a cue to prefetch from a remote peer; also you
_might_ want to keep type and size cached if you plan to implement
forgetful storage that deliberately loses objects and expects to refetch it
from its peers. Good test program once you are done extending this part is
git-cat-file with -t and -s flag.
The above three are the primary read interfaces, but there are some places that
cheat by assuming that packs and individual objects are the only two kinds of
sources for the object data, so you need to be careful. For example,
write_sha1_to_fd(), which is used only by ssh-upload, first tries to call
map_sha1_file_internal(), which is only valid for individual objects, to grab
the object data, and when it fails, calls read_packed_sha1(), which is only
valid for objects in packs, without even checking if read_packed_sha1()
succeeded. This doesn't crash only because the caller of write_sha1_to_fd()
checks if the object is available by calling has_sha1_file() itself before
calling this function, but you would need to change it to fall back on your
networked storage if you do not want to crash when used by ssh-upload.
HTH, and have fun.
Jeff thanked him, adding, "It looks pretty straightforward to add a new storage
backend, grepping on the symbols you listed."
7. Fixing Incorrect Authors And Committers In git Repositories
4 Nov (3 posts) Archive Link: "Fixing Commit And Author"
Topics: Compression
People: Darrin Thompson, Linus Torvalds
On the git mailing list, Darrin Thompson said:
I've got a small project in git where I made a dumb error. All my commits have
author/committer information like this:
Author: Darrin Thompson <darrint@dhcp-1-211.(none)> 2005-10-20 16:50:38
Committer: Darrin Thompson <darrint@dhcp-1-211.(none)> 2005-10-20c16:50:38
Tags: svn-5099
I'd like to replace the commits (yes, I know that means all of them) with new
ones with corrected email addresses and also manage to migrate my tags. A push
in the right direction would be appreciated.
Next I'd like to do the same with the kernel sources... :-)
Linus Torvalds replied:
There's a program in the git sources called "git-convert-objects.c".
It basically knows how to walk the git object chains, and rewrite each object
according to a few rules.
The rules currently do _not_ include changing the author/committer info, but it
does know how to parse the really old-style dates, for example, which are on
those same lines, so adding some code there to also re-write the author/
committer name and email wouldn't be impossible.
The code isn't necessarily all that easy to understand, and usage-wise you also
have to convert each head separately (you tell it which branch head you want to
convert, it trawls every reachable object from that head, and will create the
new objects and return the new head value).
What I'm trying to say is that it might not be _pleasant_, but it's certainly
something you can automate and do in a timely manner (ie a small project will
take just a few seconds - or minutes - to convert).
As far as doing this for the Linux kernel, Linus added:
The same program will work, but it will take some time.
Actually, as long as you only rewrite commits, it should even be reasonably
efficient. It's when you start rewriting every single object (like I did when I
switched the compression scheme around) that it gets _really_ expensive, and a
project like the kernel would take a long long time.
Hint to the wise: don't do the conversion on the only copy of the repository
you have. It's always worked for me, but hey, maybe I'm just lucky and never
write buggy conversion software.
8. Ubuntu Kernel Maintained In git
5 Nov - 9 Nov (18 posts) Archive Link: "[ANNOUNCE] Ubuntu kernel tree"
People: Ben Collins, Pavel Machek, Greg KH
Ben Collins said:
Some people may have noticed the new git tree located at:
rsync.kernel.org:/pub/scm/linux/kernel/git/bcollins/ubuntu-2.6.git
This tree will directly reflect the Ubuntu Linux Kernel that is available in
our distribution (along with build system). First use of this kernel tree is
slated for Dapper Drake (Ubuntu 6.04), and will stay synced with the just
released 2.6.14(.y).
There are several reasons for making this repo available on kernel.org. Primary
reasons include a more open development model, better visibility with the
kernel developer community, and to make the kernel available to other distro's
who may want to base their kernel off of ours.
Primary goals include:
* A kernel geared toward a real world Linux distribution, supporting drivers
and subsystems that end users need. You will find a lot of external drivers
in our tree, that for whatever reason, are not included in the upstream
kernel. We hope that including these drivers will give users a one-stop
kernel (no downloading and compiling external modules), and also provide
much needed testing for modules hoping to be included into the mainstream
kernel.
* Real world configurations. We will provide default kernel configs for a
variety of architectures and system "flavors".
* Any feature and/or driver included will attempt to be configurable. That
is, if you don't select to compile it, it will not cause any significant
changes from the stock kernel we are using at that point.
* Open development model. We want to be as close to the kernel community as
possible. Integrating ideas, getting feedback, and causing as little havoc
as possible :)
Pavel Machek remarked, " We were thinking about using git for internal suse
kernel trees, but we thought it would not work, as we need more something like
quilt. Do you really use git internally, or do you just export to it? If git is
usable for distro develompent... that would be good news." Ben replied, "Prior
to this we were using baz (arch derivative), but all the patches were still
applied at build time." He went on, "We're giving git a go just to see. It's
all being done right there, I push directly to master.kernel.org (and also
mirror it to a local machine where ubuntu devs can make better use of it)."
Greg KH remarked, "good luck with this, it will be interesting to see how long
you can handle maintaining the kernel in this manner."
9. git 0.99.9e Released
6 Nov - 7 Nov (5 posts) Archive Link: "GIT 0.99.9e"
People: Junio C. Hamano, Junio C Hamano, Johannes, Josef Weidendorfer
Junio C. Hamano announced git 0.99.9e, saying:
GIT 0.99.9e maintenance release is found at the usual places:
RPM, tarballs, and deb:
http://www.kernel.org/pub/software/scm/git/
With git, fetch maint branch from
git://git.kernel.org/pub/scm/git/git.git/
It contains everything from the master branch. Since we seem to be shelving the
separate git binary directory idea indefinitely, what we have here is pretty
much what will be in 1.0, from the source code POV.
* http-push seems to still have a bug or two but that is to be expected for
any new code, and I am reasonably sure it can be ironed out; preferably
before 1.0 but it is not a showstopper.
* I've done the initial round of package splitting for Debian side myself,
but it probably needs proofreading and fixing by experienced Debian person.
Similar RPM package splitting that parallels the above is needed. Although
it is not an absolute requirement for my sources to have perfect binary
packaging support (I am just an upstream for binary packagers), it is
certainly desirable to have RPM specs and debian/ files in a presentable
shape for 1.0.
* I still need to go over the tutorial and core-ish documentation once for
consistency checks.
He posted the changelog:
Changes since 0.99.9d are:
Johannes Schindelin:
Allow GIT_DIR to be an absolute path
http-fetch: do not use curl_message after releasing it
Jon Loeliger:
Refactor merge strategies into separate includable file.
Junio C Hamano:
test: t4102-apply-rename fails with strict umask (Peter Baumann).
git-format-patch: silly typo fix.
Documentation: pull/clone ref mapping clarification (Josef Weidendorfer).
git-fetch: fail if specified refspec does not match remote.
Simplify CFLAGS/DEFINES in Makefile
Package split: Debian.
Install asciidoc sources as well.
Further Debian split fixes.
Debian: test build.
Merge in http-push first stage.
Document expat dependency when using http-push.
ls-files: --others should not say unmerged paths are unknown.
git-status: do not mark unmerged paths as committable.
Set up remotes/origin to track all remote branches.
Nick Hengeveld:
Add support for pushing to a remote repository using HTTP/DAV
Verify remote packs, speed up pending request queue
Support remote references with slashes in their names
Improve lock handling
Refresh the remote lock if it is about to expire
Paul Collins:
http-push.c: include with angle bracket, not dq.
Randal L. Schwartz:
Use fink/darwinport paths for OSX
10. New Trivial Patch Monkey Maintainer
7 Nov (3 posts) Archive Link: "[2.6 patch] GIT trivial tree"
Topics: Version Control
People: Adrian Bunk, Linus Torvalds
Adrian Bunk said:
Linus, please do an update from:
http://www.kernel.org/pub/scm/linux/kernel/git/bunk/trivial.git
I've taken over the trivial patch monkey from Rusty, and I'll send the really
trivial patches like spelling corrections through this tree.
Is this tree OK or are there any problems with it?
Linus Torvalds replied:
Please don't try to make me update over http. Either point to master (which is
not accessible by all), or point to git://git.kernel.org/. Or do both..
And if you do the latter (or, in fact, rsync or http for others), please make
sure that you delay your "please pull" sufficiently that the contents have
actually mirrored out, because otherwise, if the mail comes in while I'm in
merging mode (like right now), and I try to pull, I may not have anything to
pull at all just because it hasn't mirrored out yet.
A side comment (this was true with BK too): I prefer not to see unnecessary
two-way merges, since that just makes the history much messier.
11. Another Attempt To Remove DevFS
7 Nov (1 post) Archive Link: "[GIT PATCH] Remove devfs from 2.6.14 - try 2"
Topics: FS: devfs
People: Greg KH
Greg KH said:
Here are the same "delete devfs" patches that I submitted for 2.6.12 and
2.6.13. It rips out all of devfs from the kernel and ends up saving a lot of
space. Since 2.6.13 came out, I have seen no complaints about the fact that
devfs was not able to be enabled anymore, and in fact, a lot of different
subsystems have already been deleting devfs support for a while now, with
apparently no complaints (due to the lack of users.)
Please pull from:
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/gregkh/devfs-2.6.git/
or if master.kernel.org hasn't synced up yet:
master.kernel.org:/pub/scm/linux/kernel/git/gregkh/devfs-2.6.git/ (http://
www.master.kernel.org:/pub/scm/linux/kernel/git/gregkh/devfs-2.6.git/)
I've posted all of these patches before, but if people really want to look at
them, they can be found
http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/gregkh-05-devfs
/
Also, if people _really_ are in love with the idea of an in-kernel devfs, I
have posted a patch that does this in about 300 lines of code, called ndevfs.
It is available in the archives if anyone wants to use that instead (it is
quite easy to maintain that patch outside of the kernel tree, due to it only
needing 3 hooks into the main kernel tree.)
12. Secure Digital Host Controller Support
8 Nov (1 post) Archive Link: "[ANNOUNCE] Driver project for Secure Digital
Host Controller Interface"
People: Pierre Ossman
Pierre Ossman said, "I've started working on a driver for the Secure Digital
Host Controller specification. This seems to be used by the majority of the
controllers found in todays PCs, so this driver would add support for a lot of
chips. Information is scarce so I do not know if this driver will get
production ready, but for those of you willing to live dangerously: http://
mmc.drzeus.cx/wiki/Linux/Drivers/sdhci"
13. Linux 2.6.14.1 Released
4 Nov - 9 Nov (19 posts) Archive Link: "Linux 2.6.14.1"
People: Greg KH, Alexander Viro
Greg KH announced Linux 2.6.14.1, saying:
We (the -stable team) are announcing the release of the 2.6.14.1 kernel.
The diffstat and short summary of the fixes are below.
I'll also be replying to this message with a copy of the patch between 2.6.14
and 2.6.14.1, as it is small enough to do so.
The updated 2.6.14.y git tree can be found at:
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/gregkh/linux-2.6.14.y.git
and can be browsed at the normal kernel.org git web browser:
www.kernel.org/git/ (http://www.kernel.org/git/)
He posted the changelog, which had only a single entry, a fix from Alexander
Viro for a "CVE-2005-2709 sysctl unregistration oops" .
14. Linux 2.4.32-rc3 Released
9 Nov - 10 Nov (6 posts) Archive Link: "Linux 2.4.32-rc3"
Topics: Ioctls
People: Marcelo Tosatti
Marcelo Tosatti announced Linux 2.4.32-rc3, saying:
Two small issues showed up,
* IPVS refcont leak
* unpriviledged virtual terminal ioctls should be allowed to read function
keys
So here goes -rc3.
Attaching the full changelog since this is probably the last -rc release.
Sharon And Joy
Kernel Traffic is grateful to be developed on a computer donated by Professor
Greg Benson and Professor Allan Cruse in the Department of Computer Science at
the University of San Francisco. This is the same department that invented
FlashMob Computing. Kernel Traffic is hosted by the generous folks at
kernel.org. All pages on this site are copyright their original authors, and
distributed under the terms of the GNU General Public License version 2.0.