...making Linux just a little more fun!
Hi Gang, I asked another question around one month ago and even though I could not come up with a answer I decided to come up here again.(Incidentally I tried it also in linuxquestions.org). What would be the equivalent to the zsh script given below in bash
$echo ${${(z) $(whereis libcrypto)}[2]}
if the output of $(whereis libcrypto) is
libcrypto: libcrypto.so libcryto.a it returns libcrypto.so only.
Now I could come up with
$robin=($(whereis libcrypto)); echo ${robin[1]}
But can it be done in one go using some construct? I am not a shell guru so I wonder if it can be done!
Robin
Dear God, yahoo really b0rked that one. All because I wanted to underline .net... Sorry 'bout that.
--I've been vaguely interested in clustering for a while, but really don't know what I could do with it. I have 3 machines:
--I can't recompile a kernel intended for the slow Intel machines on the fast AMD, it just doesn't work. However, I could go out and buy some cheap machines at a computer show for ~$30-$40 apiece, or get some loaners from a friend.
--My question is, what are clusters good for besides graphics / animation / video editing / number crunching (none of which I'm interested in)? Can I gzip/ bzip2 compress a 4-8GB tar file on a distributed cluster and save time? Can I recompile a 2.4 kernel that way so it doesn't take an hour and a half? (Yes, even using "make -s -j 3".)
--I've heard of OpenMosix, but haven't looked into it very far. Just a note, my network is 100MBit Ethernet. Any info the LG.net folks could provide would be welcome, TIA.
===== Contents above ThisLine (C)ThisYear KingNeutron Ltd.
[Heather] We'll need permission to publish your message and responses related to the thread for the world wide webzine Linux Gazette, it you want a decent chance at an answer.
This is clear enough to post as a Wanted if none of the Answer Gang want to take a pop at it, but we're past deadline for the current issue; it'd end up in February.
And of course, he did grant that, and here you have it. But it's worth noting to folks who copyright their usenet posts, we need your intent to be public or we can't publish it :) And if we can't publish it, we often won't answer it either.
===== Check out KNOPPIX Debian/Linux 700MB Live CD: ===== http://www.knopper.net/knoppix/index-old-en.html "C00K13 M0N573R 0WNZ J00!! PH34R C00K13 M0N573R 4ND 0SC4R 4ND 3LM0 4ND 5NUFFL3UP46U5 4ND 7H3 31337 535AM3 57R337 CR3W!!" .dotgoeshere.
[Heather] To the tiniest portion of an answer: there's a mosix aware variant of Knoppix. Check out LWN's Distributions page. (lwn.net/Distributions) If you get things going, I agree wholeheartedly -- this could make a really fun article by you about getting your cluster going. Are you suggesting that you could write this, or that you would like to see the topic come up?
As such I've changed the gazette@ (main editor) cc to articles@ (article ideas and submissions)
[Ashwin] Have you tried Cluster Knoppix?
http://bofh.be/clusterknoppix
Dear Ben, hello.
My name is Juan Carlos Diez, a novice Unix Sysadmin with no experience at all with sendmail, who needs desperately your kindly help
I read your sendmail notes on http://linuxgazette.net/issue58/okopnik2.html and I thought maybe you could help me.
Currently we have a Red Hat v6 server with sendmail v 8.9.3 running. We have set a new server with Red Hat 7.3 and sendmail 8.11.6 in order to migrate all of our services from the current server to the new one.
My question is: May I use the current sendmail.cf and sendmail.mc files in the new server to avoid manually configuring sendmail again? I mean, to copy such files to the new server and restart sendmail, do you think it will work fine?
Thank you very much.
[Thomas] Why have you not tested it yourselves? You certainly have nothing to loose by doing so To answer your question though, you will have no problems using the configuration files. They're based in a rather interesting scripting language called m4 which is a separate entity to sendmail.
[Sluggo] He may not know what TAG is, Thomas.
TAG is The Answer Gang, the group of volunteers at Linux Gazette that answers tech-support questions. All questions and answers are considered for publication in a future issue. Please address follow-ups or future questions to [email protected]. More information about The Answer Gang is at: http://linuxgazette.net/tag/ask-the-gang.html
Thank you all for answering, I really appreciate it.
Best regards,
JCD.
BTW, thanks for your comments on the article. Your editorial
efforts make LG a better magazine.
Editorial oversight does matter. People will see that in the magazine's quality.
- Rob
I would disagree with a softening of the advise never to log in as root. Respectively, I would remind that there are two types of Linux/Unix Administrators --
1. Those who have trashed the entire system by mistake.
2. Those who will trash the entire system some time in the future.
Personally, I avoid logging in as root as much as possible, preferring to su for a brief perioed when necessary.
Kind regards
Tony Dearson
[Sluggo] I don't see much difference between logging in as root vs su'ing. Root is root.
[Ben] Ever dip your finger in liquid nitrogen, Mike? It's a geek thing, FSVO "geek" (e.g., radars and black-body targets.) In-and-out, no problem. Keep it there for a full second or so, and you'll have frostbite that may require amputation. (Five seconds or so, and you'll be able to shatter it like glass.) When you log in as root, everything you do - and the chain of consequences that proceeds from it - is done as root, and you must consider that consequence tree for every single command you type including "ls". When I type a command, then realize that I need root privs for it, I just hit the up-arrow, type:
"<Ctrl-A>su -c "
think about the effects, and press 'Enter'. Somehow, typing 10 characters (7 if I know it beforehand) does not seem to me to be the Sysyphus' burden (with a bit of Prometheus thrown in for effect) that you insist it is.
[Sluggo] I hate typing "su[Enter][password][Enter][command][Enter][ctrl-D]" all the time, or even worse, "su -c 'command in quotes'[Enter][password][Enter]".
[Ben] [shrug] Everyone has their pet hates, of course. This does not in any way correlate to what makes for reasonable system usage.
[Sluggo] I love the fact that konsole has a "Root console" menu option, so I can get a root session any time without using the arcane su syntax.
[Ben] In what way is it arcane? Is "-c" in some way connected to the Norse Edda and the Boghaz-keuy Babylonian tablets? I wasn't aware. Molehill -> mountain requires a sufficient amount of dirt, and I'm afraid there just isn't enough.
[Sluggo] (You do have to type the root password, of course.) Even with that, I usually leave the session open in the background for a while in case I need it again.
The most important thing I ever learned (from sysadmin Pann McCuaig if he's reading) is, sit on your hands before pressing [Enter] after any potentially destructive command like "rm -rf", "rsync", etc.
[Ben] [grin] I prefer to use my other end to control my hands when using root privilege, thanks.
[Sluggo] Make sure you're the right user, on the right system, and in the right current directory. (Actually sitting on your hands is optional, but the metaphor is good so you don't forget it.)
If all that information doesn't appear in your shell prompt, read the manpage for your shell and set PS1 or PROMPT accordingly. My favorite prompt strings for zsh are:
PROMPT="%S%n@$HOST_:%~%#%s " PROMPT2='%S%_>%s '
For root I use bash with these lines:
export PS1='ROOT@`hostname`:`pwd`$ ' export PS2='> '
[Ben] I've done the following in my "~/.bashrc":
See attached ben.bashrc.txt
Whenever I'm root, my text is all in red. Makes for a decent clue, hard to miss.
[Sluggo] (Normally people use '#' to distinguish their root prompt, but I find that too easy to miss. I also need '$' to show it's bash rather than zsh (whose conventional symbol is '%').
Another important thing to remember is that "su -" gives you root's (or anybody's) full shell environment,
[Thomas] ...so that $SHELL, and other ~/.profile files, etc are sourced, as well as various other exported $VARIABLES are updated.
[Sluggo] while "su" alone gives you a partial environment that doesn't include all their environment vars. E.g., "echo $USER" (or "echo $LOGNAME" in some shells) shows your login rather than root's. That may cause some programs to do the Wrong Thing; e.g., 'mutt' will read your mail rather than root's.
[Thomas] Will not change $SHELL, and other variables.
This can be negated with the:
su -m
switch.
[Sluggo] (No, you still shouldn't read mail as root anyway! Put those dark classes back in your pocket, Ben.)
Mike,
That's a good tip, thanks. A small detail I've never come across before
Tom
Privately sent to Ben, and forwarded to the Linux Gazette for publication. -- Heather
Dear sir,
I am using Red Hat 7.2 and facing same problem after installing sendmail. Might I have made mistake while confugering Sendmail. But I am trying to find the mistake.
I got a lot Encouragement from ur efforts.
Thanking you.
With Regards,
Sam
[Thomas] Are you saying that you followed Ben's article (http://linuxgazette.net/issue52/okopnik.html) and that after you installed sendmail the problem arose that /bin/bash access denied? That being the case I would either do:
rpm -qilp sendmail
to generate a list of files belonging to that package, and check the perms for each (assuming you're using an RPM-based distro), or under debian:
dpkg -L sendmail
I am deliberately being vague here.
Your question as it stands lacks structure, information and meaning. I suggest you look here:
http://linuxgazette.net/tag/ask-the-gang.html
for further information.
-- Thomas Adam
I'm not sure where this should be shoehorned into our FAQs, but - having just read this excellent 25-page paper, I think it should be required reading for prospective authors (or anyone who wants to write a technical paper). "Clarity in Technical Reporting", which had a long run as an underground publication at NASA, was officially published by The Powers That Be once they caught on (thus proving that they were worthy of their positions. All hail, etc.)
http://techreports.larc.nasa.gov/ltrs/PDF/NASA-64-sp7010.pdf
Hi all. Just grabbed the #97 and am having some problems reading it on Mozilla 1.6b on Mandrake 9.0. The navigation box overlays the first several lines of each article. I played a bit with lg.css (which I don't know anything about) and got it readable by outcommenting the line:
position: absolute;
at line 53.
Now, the article text overlays the nav. box. Not what you intended, but it is readable... a real fix would be appreciated. Hey, this bug might even spur me on to learn how to use stylesheets
And: keep up the great work!
[Mike] Thanks. Our stylesheet editor, Rob Tougher, is away on family business, and I have only a rudimentary knowledge of CSS. Are the menu links showing up one per line with a black bullet left of each? And the breadcrumbs (=Yahoo bar) too? That was something I specifically fixed yesterday, because our stylesheet fixes this month (for Netscape 4 and Phoenix) broke compatibility with issue 97. If you do "View Source" on an offending page and scroll to the bottom, the menu links should not have <li>...</li> around them. If they do, your browser is caching an obsolete version of the page.
I tried your solution but it breaks in my Galeon: it moves the menubar to the very top, covering part of the logo and Tux. While this is readable, it's not acceptable. Thanks for letting us know anyway, and for any other ideas you might have later.
A new problem we have is that after I added the "Contact Us" link, the menubar is extending too far to the right past Tux. I couldn't figure out how to prevent that. Do you have any ideas?
Well, problem is solved. I dl'd the version on the site just now and it works just fine. Guess I grabbed while you were putting, or something. Thanks!
Heather,
Been reading for a while and lost the site when you started having problems. Nice to find it again!
Keep up the good work.
Martin
Thanks, Martin, I'm glad you like how we've kept it. Happy New Year -- Heather
This is a multi-part message in MIME format.
You'd think that if Sanjaya had been on this list before that he would
have remembered that none of us appreciate MIME encoded e-mails.... oh
well...
I guess a beneficial reading of:
http://linuxgazette.net/tag/ask-the-gang.html
is in order. -- Thomas
I used to be on the tag mailing list some time ago. Does it still exist? How can I join it? thanks.
[Mike] http://linuxgazette.net/mailman/listinfo/tag
We couldn't carry the subscribers over from the old linux-questions-only list because SSC wouldn't give us the member roster. Now the list is fully under our control.
[Breen] By the way, I'm back too. It's been pretty busy IRL, but I finally got around to moving here.
Cheers and Happy New Year,
doesn't seem obvious to find http://linuxgazette.net/mailman/listinfo/tag on the home page of http://linuxgazette.net Did I miss something?
Wishing all a happy new year!
sanjaya.
[Ben] It's not supposed to be obvious; you were just shown a shortcut. If you want to join, the what you're really supposed to do is read the TAG FAQ at:
http://linuxgazette.net/tag/members-faq.html
which will direct you to the subscription address, etc. after explaining the necessary facts of life - such as, you do not need to subscribe to TAG in order to ask a question. Subscriptions to TAG are for those who are willing and able to answer Linux questions.
Good point - we should update the "About TAG" blurb on the mailman link given above to include this concept, give the hotlink to member policy, and advise querents to read ask-the-gang and then just send mail. -- Heather
Thanks. We've been debating how much to change the back issues. Plus it's a big job, and we can't change the mirrors that don't resync back issues.
Hello Mike,
No worries. Just letting you know, what you do about it is up to you.
"The Wonderful World of Linux 2.6" is absolutely incredible. What a great article.
cheers
Many new laptops come without a floppy drive; the one that I just bought, an Acer Aspire 2003LMi, does not have one - it's available as an extra option, but I don't see myself needing it. However, part of my standard method for converting Win-machines to dual-boot involves using FIPS, which I use to "shrink" the Wind0ws partition to a minimal size - and FIPS normally runs from a floppy. What to do?
Here's an interesting fact that many people may not be aware of: the bootable part of a CD consists of nothing more than a bootable floppy image. So, I simply took a DOS boot floppy containing FIPS, and made a byte-by-byte copy:
# Create a directory to hold the CD data (none at the moment) plus "boot" ben@Fenrir:~$ mkdir -p /tmp/cdrom/boot
# Create byte-by-byte copy in "boot.img" ben@Fenrir:~$ dd if=/dev/fd0 of=/tmp/cdrom/boot/boot.img
I then created an ISO image containing that disk copy. If I wanted any other data on that CD (I'll probably make another one with a bunch of DOS utilities on it later; I've been using bootable DOS "tool" floppies for over 20 years to repair broken Wind0ws systems), I'd copy that data into "/tmp/cdrom", and it would become part of that image.
ben@Fenrir:~$ cd /tmp/cdrom ben@Fenrir:/tmp/cdrom$ mkisofs -r -b boot/boot.img -c boot/boot.catalog -o bootcd.iso .
All that was left was to burn the newly-created image to a CD:
ben@Fenrir:/tmp/cdrom$ sudo cdrecord -v -eject speed=8 dev=0,0,0 bootcd.iso
The only downside to this is not being able to save the boot sector to the floppy before repartitioning... but in the worst case, it's a new system without any of my data on it, and it's not a concern. Besides, I have Linux, and boot sector recovery is rather trivial.
Hi, first of all, sorry for my english, I try my best, but I use to fail in spelling and grammar Well, I'm writting to you because I have a problem at Xfree86's startup. I'm useing kernel 2.4, so I have, in Xwrapper.config, nice value set to -10 ("nice_value=-10"), but when entering "startx", just before entering Xfree86, a message is shown: "warning: process set to nice value -11 instead of -10 as requested". If I then change nice value to -11, then the startx script changes nice value to -12. If I change it to -12, it changes it to -13, and so on for any number between [-20, 19]. I've searched for many days (weeks, in fact) on the web and asked at #debian, but no answers. I hope you can lend me a hand. Btw, XFree86 runs perfectly ok, it's just that I hate having error messages. Thanx a lot for your help!
[Thomas] By my knowledge of how X starts up it is not startx which is changing this. Just out of curiosity, run (as root):
dpkg-reconfigure xserver-common
to see if that fixes anything. I have grepped through the startup files that I use (I am running debian unstable) and there is nothing besides the value in /etc/X11/Xwrapper.config which sets or changes the nice value of X.
As a long shot (and possibly a complete aside) you don't have the "and" package installed, do you?
Upto how many CPUs does Linux Support in --
a) CISC Technology
b) RISC Technology
[ashwin] The numbers for the individual architectures for the current kernels can be got here - http://www.tldp.org/HOWTO/SMP-HOWTO.html
Expect much much better support with the 2.6 series.
Can I load linux onto this machine? Where do I get the software? thanks
[Neil] MkLinux supports the 5200 see http://linuxtoday.com/developer/2000080401404OSHWKN
- Other Mac Linuxes include Yellow dog Linux
- http://www.yellowdoglinux.com/support/hardware/breakdown/index.php
How do I get an E-maikl to above subject? They interrupt my messages.
[Jason] What you seem to be asking is "why am I getting messages from Mailer-Daemon"? A message from "Mailer-Daemon" is probably the MTA (Mail transport agent: The software that delivers mail.) sending a message that says, in effect, "excuse me, but you seem to be confused. There is no such mailbox here."
claim no such addresses exist, etc.,
[Jason] The automated message you're recieving means just what it says: No such address exists. For example, if you try to send email to [email protected], and there is no user bob at example.com, you're probably going to get a bounce message that says no such address exists.
yet they cannot be questioned or challenged ever when they are wrong. Help.
[Jason] The reason "they" cannot be questioned is because you're getting an automated message: It was not send by a human.
Nobody would know better whether an address exists than the system you're sending mail to, so I would say that the system you're sending mail to is right and you are wrong.
Now -- it's an unbearable situation that my Linux doesn't know "no".... But I've no idea what it is and you can imagine that a google for "no" even with linux and some other keywords around are not very helpful.
Any ideas?
[Faber] Maybe it's called "nein" on your computer? <grvf>
I can't find a "no" on my Red Hat 8 box either.
[JimD] I think /usr/bin/no was (would be) a counterpart to the old /usr/bin/yes command:
#!/bin/sh OUTPUT='y' [ "$#" -gt 1] && OUTPUT="$*" while : ; do echo "$OUTPUT" done
... so "no" could just be an alias or script that calls /usr/bin/yes with the "no" argument:
/usr/bin/yes no
I realize this sounds silly and stupid, and April 1st is long past for this year. But I'm not kidding. That Makefile (or whatever) seems to actually want to pipe an endless stream of "n" or "no" lines into some other process. (/usr/bin/yes was traditionally used in a pipeline with fsck to automate the process of repairing a filesystem that need lots of work -- then they just added the -y option to the GNU/Linux versions of fsck.
I put the question up with bugreports for binutils and got:
[Alan]
> /bin/sh: no: command not found
This is a result of binutils being stuck on using old buggy autoconf. Install a new version of GNU gettext, or configure with --disable-nls.
-- Alan Modra IBM OzLabs - Linux Technology Centre
Got a new gettext which includes some "no"'s
khh > find ./ -name "no*" ./gettext-runtime/po/no.po ./gettext-runtime/po/no.gmo ./gettext-tools/po/no.po ./gettext-tools/po/no.gmo
unfortunately with a new gettext (gettext-0.12.1.tar.gz) and nls enabled I get a linker error for some gettext symbol. The solution without nls works for getting binutils compiled. I try that on the 2.5.70 kernel sometime soon.
Does anybody know of a backport to 2.4.X of the preempt patch and or the I/O scheme patches mentioned on kerneltrap right now? I'm not yet sure what else will break if I switch to 2.5.X. At least NVdriver, lt_serial+lt_modem and vmware kernel modules would be nice to have.
[Thomas] Well, she's famous for being married to Linus Torvalds.
[Ashwin] She is also famous for being Finland's champion in kung-fu or some similar martial art
Linus was interviewed in Issue 67 of Linux Journal, by Marjorie Richardson at the Linux World Expo of that year. Tove was 6 years running, the Finnish champion for karate, specializing in precise Kata (the forms), then moved on to other interests. -- Heather
Hi all,
I use kppp under linux to dial-up my isp. but there is a strange problem happens with me. when I dial-up from windows98 it connects with the isp at the very first attempt. but under Linux (debian woody) kppp takes at-least 3/4 attempts to connect the isp, and during the failure it shows *pppd can't be started * . I have also started pppd from root manually at the time of hooking, but the result is same. could some one please suggest me how to fix the problem ?
thanks in advanced.
[Thomas] You need to ensure that you have a ppp-chat script enabled which is used by pppd to comminicate with the modem to send certain signals, etc.
wvdial will help for this
Hi,
I'm running red hat 9 with an ethernet card to a LAN and a modem for dial up. I'm using kppp for the dial up. When the eth0 int is active, kppp will establish a connection but DNS fails. The DNS listed for both interfaces is the dialup one, but I can't resolve names unless the eth0 is shutdown. How can I , an ordinary joe get these things to work at the same time ?
thanks,
cj
[Thomas] There is either a really easy or hard explanation. I think I am right in saying that you need to have a correct entry in your routing table to use the two concurrently, since the routing will not know.
As far as DNS goes, do you have a valid entry in /etc/resolv.conf and also an entry in /etc/nsswitch.conf:
hosts: files dns
like that? If not, add the "dns" after the word "files".
[K.-H.] Well kppp (or pppd which kppp calls) refuses to setup a default route if one already exists. The magic scrying ball (glass?) would suggest with eth0 up you've a default route set. On dailup via kppp you won't get a default route to your ppp0 interface and therefore DNS lookups to the world outside never reach there.
This is how it should look lie with both eth0 and ppp0 (kppp) up (I cut out three columns which are unimportant):
# route -n Kernel IP routing table Destination Gateway Genmask Flags Iface 62.104.218.38 0.0.0.0 255.255.255.255 UH ppp0 192.168.2.3 0.0.0.0 255.255.255.255 UH dummy0 10.10.10.0 0.0.0.0 255.255.255.0 U eth0 172.16.57.0 0.0.0.0 255.255.255.0 U vmnet1 0.0.0.0 62.104.218.38 0.0.0.0 UG ppp0
I've two private networks, real ethernet as 10.10.10.0 and vmware virtual host as 172.16.57.0. The "0" at the end suggests and a genmask of 255.255.255.0 proves that these are networks, i.e. not a single host but all hosts 10.10.10.x with 1<x<255 (zero is broadcast address to all). If you look at the flags "H" means this is only one single host, "G" means this is a gateway.
The last line is the default routing, i.e. if no other routing rule applies all remaining traffic this way. The way is interface ppp0 and the target the gateway IP 62.104.218.38, our remote host on the other end of the modem line (see first line).
Now I'm pretty sure that in your case there is a line like:
0.0.0.0 [some IP] 0.0.0.0 UG eth0
If this is the case kppp will not touch it and if you would have looked carefully in your /var/log/messages (or kppp's log) you would have found an error telling you this.
If I guessed right run (as root):
route del default
then start kppp's dialin
After that figure out why RedHat sets a default route (I'm with the lizard and without hats).
If I did not guess right you oviously didn't give enough information....
Hi Answer Guy,
I am having a peculiar problem with the Red Hat 9 installation. I am trying to install it on a IBM thinkpad 1721 laptop with a formatted hard drive. The laptop has a combo floppy/CD ROM drive. The laptop seems to be seeing the CD ROM dirve but it is not able to read anything off it. The laptop has a PCMCIA network adapter card which I used to connect to my home networking. The other laptop on the home networking has Win 2000 running on it. Here are the following waya I tried to install Linux -
1. I created a Linux boot disk using the pcmcia.img and booted the IBM laptop and selected NFS install. On the other laptop I copied all the Red Hat folders from the 3 installation CDs. I selected "Automatic DNS configuration" to configure the DNS names for the IBM machine. For the NFS server name I typed in the name (which did not have a domain name since I log into a workgroup and not a domain) and typed in the actual directory of the Redhat parent folder (d:\redhat9). But I get an error that the drive could not be mounted.
Then I grabbed a Freeware called NFSClientServer and installed it on the Win 200 laptop. Here I exported the RedHat directory (d:\RedHat9\RedHat). Then I followed the above steps on the IBM machine for NFS installation. Even though my requests form the IBM machine seemed to log in the Server log on the Win 2000 machine, I still could not mount the directory. What am I missing?
2. Failing the above attempt, I attached a SCSI CD ROM to my IBM machine. Then I made a boot disk using the bootdisk.img. After booting the IBM, I selected "local CDROM" for the source, but I kept getting the message "No software found in CDROM". Obvously, Linux was not seeing my SCSI CDROM, but since it detected my internal (failed) CDROM, and could not read off of it, it was giving me the message. How do I make it look at my SCSI CDROM?
I even tried making a boot disk using the "drvblock.img" file, but for some reason when I use the disk, I get the message " No operation system found". When I look at the disk contents the, it seems like the format information on the disk is lost and Windows explorer asks me if I "want to format the disk"? I even tried the same using Linux 7.3 (Red Hat), same error.
Please help.
Regards,
A Wexed Linux Installer.
[Mike] OK - your best bet would be a HD install. If you have copied the folders over you have enough HD space.
In w2k make a directory to hold the iso images In turn copy the cds to this directory (not the contents - I think Nero should be able to do this) You should then have three files in this directory, all ending in .iso
Now boot using your boot.img disk. When you get get prompted for installation type, choose HD then select the right partition where there are held (probably /dev/hda1 on your system if you choose the first partition) then select the directory you have placed the .iso files in.
Then you should be good to go.
Dear Answer Gang,
Try as I might, I am unable to figure out just what to do to allow a trusted user on a trusted host to use 'scp' to copy files from one system to another WITHOUT a password prompt. We WANT to do this in order to use 'scp' in scripts initiated by CRON. We NEED to do this because 'rcp' gives us 'file too large' responses. I have tried modifying /etc/pam.d/sshd (at least to the limits of my understanding). I can make scp NOT work at all, or require the password, but I cannot make it work without the password. Changing the /etc/ssh/sshd_config file doesn't make any difference, that I can see, even though that file now says 'go ahead and use the rhosts files'.
Can someone help me? Anyone? I would offer a carrot, like "I'll subscribe to LJ", but I already do. I would offer virtual hugs and kisses, but the respondent might be male, in which case an old homophobe like me is going to have a problem. How about "undying gratitude"? Yeah, that's the ticket ... it's easy, it's cheap, I could even teach my kids to sing appropriate praises .. yeah, that's it .... Where's Jon Lovitz when you really need him?
Lou Lohman
Don't just BELIEVE!! Consume information like a starving person, and then sort it out for yourself.
And we know he reads ask-the-gang.html, he gave us explicit permission to publish the whole thread just like we ask for Now if we could get people to remember to turn off HTML in their email... -- Heather
[Thomas] You need to run "scp" with the "-B" flag, ie.
scp -B files thomas@thomas
Batchmode doesn't require password authentication. You can also add this in /etc/ssh_config as...
BatchMode=yes
that way, you don't have to pass the -B switch each time.
[Mike] You can also exchange keys from one user/machine to the other. If you want to copy from machine A to machine B. On machine A, as the user that needs to copy, run ssh-keygen -t dsa. This will generate the key pair. Then he'll need to copy the contents of ~/.ssh/id_dsa.pub from machine A into ~/.ssh/authorized_keys on machine B. Then copying from machine A to machine B won't require a password for this user. Do the reverse to copy from B to A.
Hi Answer Gang,
I have some problems trying to connect with my HSP Pctel Micromodem 56.I have RedHat 9 installed and I use kppp for conneting to the net.When I try to connect the handshake goes on well and all of a sudden I get this "No Carrier" message.Can you tell what causes this error and what is the solution.............please sugget an alternative driver if available.At present I am using the Pctel drivers available at www.linmodems.org version 0.9.6 . Can any special AT command string help?????? If yes please suggest a solution.Or using any other dialer will help????
Thanks in advance.
Vivek.
[K.-H.] no carrier means the modem is unable to detect the carrier frequency onto which data would be encoded. As this is a fatal problem it hangs up and tells you "no carrier".
Why is the carrier gone? difficult to tell from here. What do your logfiles say? kppp has a log button, use it!
At exactly what point of connection negotiation does the error occur? Do you get the "connect" in the log window? Then the dialing is finished and control passed to pppd -- which logs it's messages in /var/log/messages or some such place. Go look for it. You can pass additional option to pppd in kppp: add "debug". Make sure to press the add button in the kppp window so the new option is actually used (should show up in the lower larger window).
My guess: serial connection gets established, pppd get's into some trouble negotiating the ppp parameters (user/passwd? pap <-> chap <-> terminal authentification, compression,.... ) and the other side terminates on you. Your modem detects the lost carrier and tells you.
Here's a suggestion for your 2 cent tips section - I hope you like it.
When working with Linux servers I don't run X, preferring to make use of the virtual terminals via Alt-F1, Alt-F2, Alt-F3 etc. The problem is knowing which screen I am looking at, so I have devised a custom prompt by including the following two lines in my .profile:
TTY=`basename \`tty\"
export PS1="[$TTY] \w$ "
Cool
The first line sets the environment variable TTY to the number of the current virtual terminal, the second sets my prompt to show that number and the current working directory as in this example:
[2] /etc/xinetd.d$
That's it!
Toby Poynder London, UK
I must admit that I often find just typing in "tty" is more efficient than having it set in one's $PS1 prompt. -- Thomas
To workaround black lines in XFree86 with Radeon mobility 7500 (thinkpad R40 here), add:
Option ""XaaNoSolidTwoPointLine""
To the drivers section of /etc/X11/XF86Config(-4)
Adrian (aka Wyvern on #hants)
- [Jason]
- http://xaos.sf.net
What is it, you ask?
Real-time fractal zoomer.
If that isn't enough to get you to run out and download it, you're probably not the kind of person who would enjoy it.
[Ben] And don't forget to take a look at the sample fractals that come with the "xaos" package, located (under Debian) in /usr/share/XaoS/examples. (I've got all of these converted to JPGs and use them as randomly-selected desktop backgrounds.) Beautiful.
[Faber] Yes, a very nice toy, but I still miss Fractint for DOS. The X/Windows version had all the features, but just doesn't cut it somehow. A Mandelbrot set not being full screen looses something, not to mention they're slower than the DOS version.
I also had an internal IBM program[1] that would map out different phase space maps stereographically, looking like some of the maps in the Discrete screen saver in xscreensaver.. Man that was a fun toy; I wish I could find the source code (it was floating around here on disk years ago); I'd try to port that to Linux.
[1] For those of you that don't know it, IBM has a a mini Internet in place with their own version of "free software", to wit programs written by Beamers and distributed to other Beamers for fun and collective profit. And since some of those people had multiple PhDs and worked on esoteric stuff, some interesting programs cropped up. It's the only thing I miss from my IBM days.
Greetings, gentle readers, and welcome once more to the world of the Answer Gang. TAG, we're it
I have to confess that I wondered to myself, what should I babble about this time? As I look at the back issues, I notice some interesting statistics... apparently, I shouldn't feel surprised that things here at Linux Gazette are a bit hectic. That's consistent with all our past Februaries. Ain't tradition grand?
February here where I sit, is often considered the month of romance. I have to admit... I (heart) Linux Gazette.
I mean, it's not like I will be buying it chocolates or wondering what its favorite cologne is. But I put a bunch of work in every month - I really like knowing you people out there are reading (hint, hint; tell us what you like in this stuff)! I love seeing what sort of curious troubles are out there to solve - the most curious, the kinds of things whose answers change over time. And it's always worth seeing what cool answers are out there.
You don't have to actually be a member of the Answer Gang to send us juicy answers, either. An honorary Answerbubble to folks who send us their good stuff - or cc us on the the juicy tip they are sending to someone anyway. When we got the Gang together, the heading mentioned a few of the active posters... and you. That hasn't changed. It's teamwork that makes this all happen, and I'm proud to be here. Thomas and I have teamed up to bring you the juiciest threads this month, and I hope they make your time with Linux this month - "just a little more fun!"
From Tom Brown
Answered By: Thomas Adam, Chaz Peters, Karl-Heinz Herrmann, Ben Okopnik, Robos
I'm trying to backup my Linux installation with tar, using a second hard-drive in my system, rather than a tape-drive or CD. The trouble is, I have a 2-GB file size limit on the destination (It's Fat32, so I can also use it for Windows backups), so I have to do it in a lot of little chunks (even with compression). Is there another solution to this, either a fancy shell script, awk script, or some combination of tar options that would produce the multiple destination files I'm looking for? If I keep doing it manually like I am now, I know I'll never maintain an up-to-date system backup like I should. I've found tape options for tar that control multivolume backups, and tape length, but nothing for multiple files.
[Chaz] Backups can be a pain, especially ones that require manual operation. I like to automate them as much as possible. The following is a script I made for Kathy's Debian machine. Usually I prefer to backup over a network to another machine, however she has dialup and no other machines on a LAN. I use rsync because it's fast and works well. rsync is a file transfer program capable of efficient remote update via a fast differencing algorithm. This program is run once a week via cron, it works very well for hassle free automated backups as long as you have enough disk space. If you require compression, this is not what you want. I do not recommend using compression for backups, compression reduces the chances that the data will be recoverable.
See attached backup-weekly.sh.txt
An example of what I'm doing now:
tar zcvf /windows/s/suse/back_tbrown.tgz /home/tbrown
Oh, I tried the Suse backup/restore function, and could not restore the resulting files. The .tar.gz files within the .tar archives (don't know why they did it that way) seem to be corrupted. So, I figured I'd do it myself.
[Chaz] SuSE, sorry the dpkg part of my script won't help...
Note the script lacks a secondary archive, that could be disastrous in a few cases. We do have an older backup on CDR and at some point I would like to transfer it to a laptop or something for other more recent off site copies. She can also selectivily transfer files via dialup so that I can back them up.
When I get more disk space, I am going to look into using better archival techniques. I have heard good things about Dirvish, a fast, disk based, rotating network backup system. A dirvish backup vault is like a time machine for your data. http://www.pegasys.ws/dirvish
[Thomas] What you can do is something like this:
(cd /src/dir && tar cf - . ) | (cd /dest/dir && tar xvfp -)
where /src/dir is the directory you're starting from, and /dest/dir is the final destination that the files (dir's) will end up to.
Since you say that this is going to a FAT32 volume, that will not preserve file permissions. The only way you can achieve that is by making a tar file.
Thanks. That's why I didn't just cp the directories over.
[Thomas] Your other option is to make an archive and burn it to CD. One thing you might want to try though, is you are going to make a tar archive, run it with the "j" flag when you create it. That'll use bzip2 and might compact some more space.
[K.-H.] You might have a look at afio instead of tar. It's more robust against data errors in the archives then tar and from reading the manpage I'm not quite sure if you can specify archive-filenames which are automatically numbered for multivolume. If not you can still automate things with the "promptscript" option. You archive to a specific dummy file, the script will mv/rename it to something useful and (number, date,...) and continue. To get rid of the prompts (or answer them automatically) should not be that difficult.
Be careful to read the basics: afio wants a list of files to be archived piped in on STDIN.
This might be a good startingpoint (no multivolumes, add that yourself):
find /var -xdev -print | afio -v -o -Z -T 5k -b10k ARCHIVE.afio
[Ben] Make your giant tarball, then use the 'split' utility to break it up into chunks. When you're ready to use it, just 'cat' all the pieces in order (which is how they'll be named by 'split') into a single file that you can untar. As someone mentioned, 'j' rather than 'z' gives you even better compression on large files.
[Heather] Since j invokes bzip2 compression, yes. I wouldn't use it if anything needs to be unpacked on a non-linux system though; other OS' are shabby at bzip2 support.
That would work fine, except that the tarball is too big to be created on the destination file system in the first place. What I'm looking for is some way of creating a lot of smaller tarballs right from the start.
[Ben] What I meant was to create it on the "source" system, not the "target" one, then split and transfer. However, you can do it "in flight", too:
tar cvzf - * | split -b 100k backup-01-15-04
[Thomas] Since the destination is not a Unix system, the use of the "-p" flag to preserve permissions is a must in this instance.
[Ben] It's not really relevant to the host OS; the permissions that matter are "inside" the tarball. However, you're right anyway - in a backup,
[Thomas] Indeed.
[Ben] permissions should be preserved, and I lost track of that in generating a random example of "split" usage. In fact, for backups, the "tar" string should be:
tar cvzpSf - *
(add sparse file handling, as well.)
[Thomas] LOL, I don't know, Ben.... all that Yoga and the like is going to your head, just make sure you:
tar cvzpSf
your linux knowledge
I for one, would be very interested in that tarball...
[Ben] Sorry, even the pieces would be too large to fit on any possible host system. Although there's a lot of sparse files there, too.
[Ben] This will create a load of 100k-sized files called "backup-01-15-04aa", "backup-01-15-04ab", etc. If the destination was a Unix system, I'd suggest piping "tar" into SSH, catching it on the far end and then_ splitting it - all done in one shot.
[Robos] I'd rather use netcat instead of ssh. Depending on the connection certainly (didn't read all). But ssh adds quite a load more to the already busy cpu which tries to to bzip compression on the fly...
I'd love to find out why the Suse backup tarballs won't untar, since Yast2 appears to do the kind of backup I want. I'm overlooking something there, I just know it, since the feature wouldn't exist in Suse if it didn't work.
[Ben] Don't know anything about SuSE backup, but the above should do what you want.
From Karl-Heinz Herrmann
[Heather] The question this time is from our own Karl-Heinz Herrmann. When you play on the edge, it's best to ask a few buddies to keep an eye out for the loose rocks...
Answered By: Robos, Rick Moen, Heather Stern
Hi Tags,
I'm hunting for new desktop hardware and getting confused and somewhat frustrated.
[Robos] Me too (sort of)
The best German computer magzine (c't) has of course plenty of tests and recommendations as well as "optimal PCs" for certain usage profiles. My problem is, I want a decent performing raid 5 (burned to often by sudden drive failure).
[Robos] Have that already in the gangway - working flawlessly for several years now. 80GB raid 5.
[Rick] Do you expect to be so incredibly short of disk space that RAID1 isn't an option?
[Heather] Funny you should mention that; I met someone recently who got burned by sudden drive failure on a RAID 1 system - and to their horror, while the other drive was fine contents-wise, it was also in a state where it couldn't be brought up normally to be looked at - expected a twin to be present, and nobody wanted to risk it considering a "mew" drive the master and wiping it. It needed a visit to the drive recovery shop.
RAID is not a panacea - a cure-all - against hardware failure; it merely helps. Usually, a lot. But it's specifically not a solution against slow corruption via software failure. Anyone working with oversized disks should seriously consider their backup-and-recovery plans too.
no -- raid1 would be an option. If some 50GB actual space are there that should be enough for some time, (temp) video data won't need raid1.
With recent consumer boards I seem to have the option of using IDE drives, sure. Cheap huge drive space and all. If I understand the board details right all of the regular ATA interfaces are on the PCI bus -- so a raid with 3 or 4 drives there is hogging the PCI bus. There are SATA interfaces which are not going through the PCI interface but have a separate one -- just Linux seems not to happy with SATA yet and most boards have 2 SATA drives -- not enough for raid5. If I could be sure that a raid1 is running stable and at full speed in Linux with SATA drives hat might be an optin, as SATA brings plenty of disk space for a reasonable price).
[Rick] Huh? RAID1 will inherently be faster than RAID5. But either is fine with the md driver.
That's not what I meant -- from all I heard SATA is still highly experimental and often not supported at all. Most often I heard as problemsolver to switch them to IDE.
[Rick Moen] That's one workaround, of three total.
I'm not quite sure if the SATA drives are connected to the IDE connections for that or if the SATA connections can be used in a compatibility mode.
[Rick] The latter -- where the BIOS supports doing so.
And I also don't know what performance changes that might bring along.
[Rick] Who cares? You use the drive in that mode only long enough to compile a new kernel with the requisite support for the SATA chipset in question.
I've been googling quite some time again and slowly I get an idea on Linux support. libata is providing sata support --
[Rick] More precisely, libata is providing better and broader SATA support.
patch in 2.4, already included in 2.6 but still under heavy development as the latest changelogs and bugfixes show.
[Rick] Correct. However, there is non-libata support for several SATA chipsets in recent 2.4 kernels. (Of those, 3Ware Escalade is supported for a long ways back in 2.4.x.)
I'm not sure about the Via K8T800 chipset
[Rick] That isn't a SATA chipset. It's a motherboard chipset -- always accompanied, to the best of my knowledge, by the VIA VT8237 South Bridge, which does SATA and is supported in libata.
but the nforce3 seems mostly supported (sound seems a dead weight).
[Rick] Nvidia nforce3 is also not an SATA chipset, but rather a motherboard one -- always accompanied, to the best of my knowledge, by a really awful Promise R20378 SATA / software-RAID chip. The sound is indeed provided by a really dreadful Realtek ALC650 chip.
I would run, not walk, in just about any direction leading away from that kit.
There are driver tar packages from nvidia for download (at least partly source, not sure if some binary only parts are in there too). The Sata controller on nforce3 boards seem to be Silicon Image like Sil3112A, Sil3114.
Robos: You said they are supported, do you have pointers for details?
[Rick] That is conceivable, but is not what I've been seeing. It's possible that it's dubious-quality Silicon Image chips in some cases, and awful quality Promise chips in others.
[Robos] Yes, my flatmates experience But he's on holiday so I can't ask him. And I don't really recall any details.
[Heather] I've got one; a buddy just compiled 2.6.1-mm4 and with a couple of stumbles making sure other parts work also, is now ready to fall the rest of the way in love with his Athlon/SATA system, bearing an sil3114 chipset. Statistics on exactly how cool not available just yet
Has anyboy here SATA drives working? Performance? kernel versions? any patches? And on what particular chipset?
[Rick] Best quality: Adaptec 24x0, 3Ware Escalade -- in that order. But you won't find those built into motherboards. Usually, what you find inside motherboard chipsets is Intel ICH5. Which is basically OK.
I don't mind AMD/P4 or even AMD64 (new 3000+ is reasonably priced).
[Robos] OK, I don't have sata drives but I read something about it. It seems (see latest c't) that all the upcoming athlon 64 boards have sata (since it's in their north|southbride) and quite a lot of them already have hardware raid0 and 1.
[Rick] You can read more about it here: "Serial ATA" on http://linuxmafia.com/kb/Hardware
Yes -- the hardware is there -- but as far as I know none of the hardware raids are supported in Linux, and booting Linux off SATA drives which run actually as SATA and not in some IDE-emulation mode is difficult. What I do not know if once a Lniux kernel takes over if they are running as real SATA? Performance penalty for IDE emulation mode? And the boards tested in latest c't all have two SATA connectors, i.e. to few for raid5. Of course I could plug in 1 or 2 additional SCSI disks on the PCI bus with them.
Alternatively I've already plenty of SCSI hardware in the old one, so I would maybe even prefer a decent SCSI setup -- but I can't find a chipset with onboard SCSI controllers -- and if I plug PCI cards I'm again hogging the PCI bus.
Are there any other chipsets besides nforce2/3, i875,....?
[Rick] Sure.
Gods above, Karl-Heinz, what sort of incredibly disk-intensive operations are you planning to conduct?
Well -- right now I've some 32% CPU at full whopping 9MB/s (bonnie 8892 kB/s 31.8 %CPU) from the SCSI raid5 in a Pent. classic board running a K6II-366MHz. Since I am hunting for new hardware I am thinking of getting something decent which I (again) can keep for some time. The old system is running since 1997 after all, with occasional low cost upgrades(like the K6).
My original idea was: Ok upgrade is necessary, IDE is still not what I want (performance trouble with my existing Lap and a DVD burner in the desktop put me off; that cdrecord mess, ide-scsi or not, DMA only with the right block size,...). So how about some new SCSI drives (3 for raid5) and a nice new card (U160 one channel should do as I still have a 2940UW for CD, scanner, etc.) and be done. Well-- the lowest grade (new) SCSI drives I can locate are some 10k rpm 18 or 36GB drives which sustain easily 30+MB/s and peak at 70 MB/s. A PCI bus is getting very much the limiting factor as there are probably more cards in there as well and I'm looking for a way to avoid building a bottleneck into the new thing right from the start. Thats why I tried to ask for info's on boards/chipsets which might have SCSI onboard, preferrably circumventing the PCI bus.
But you are probably right, just ignore the bottleneck and let 100MB/s be good enough. In a year the SATA support will be there and offers a non-PCI extension anyway if needed.
[Rick] The SATA 1.0 spec claims a 150MB/sec limit on bus transfers. Of course, the fastest ATA hard drive manufactured today can, under contrived, artificially ideal conditions, read data at maybe 50 MB/sec, so saturating ATA/100 or ATA/133 sounds very farfetched for a long time to come. (Remember, only one ATA device per bus can be active at any specific time. SATA changes this only by permitting connection of only one ATA device on the chain.
Aren't there any "server-boards" with other specifications? In the price lists here none show up, but a recent acer ad mentioned some "ServerWorks Chipset GC-SL".
[Robos] IIRC Silicon Image chips were good under linux. One of the kernel developers even managed to persuade some hw producer to donate some driver to gpl. Can't recall which one that was, might have been promise.
Any boards/manufacturers using that chipsets (available in Germany)?
[Robos] OK, I just looked on alternate.de and didn't find a silicon image but rather found this: http://www1.alternate.de/html/nodes_info/giea01.html
- Regarding their promise chipset I found this:
- http://lwn.net/Articles/40899
Can anybody point me to details what this chipset can do (http://www.acer.com doesn't)? Or knows of a nice one (like onboard SCSI [raid] controller bypassing PCI, don't forget the Linux support)? Or at least 64 bit PCI? All the consumer boards showing up in a A4 page (tiny print) of boards seem not to have anything special in that regard.
Am I overemphasising the PCI-hogging? To cite above mentioned c't -- If you run a Gigabit ethernet on PCI it basically saturates the PCI bus.
[Rick] 1. Yes.
Hmm....
[Rick] 2. Using Linux software RAID (md driver) for redundancy, you typically won't even notice the load relative to (say) a hugely expensive Mylex SCSI hardware RAID adapter except during RAID volume rebuilds (if you ever have to do that). In such a case, expect the system to very, very seriously bog down during the rebuild. With the Mylex card, all of that would be handled entirely within the array, with basically no system loading.
Every unclean reboot -- but the raid5 regeneration isn't that bad, even on the old system.
[Rick] I'm talking about when one replaces a failed ATA drive in the array. System will be usable but heavily loaded.
Takes some 10 minutes each 3-5 GB mdX. System is still usable during that time. Rebuilding is on a low prioritiy.
I picked up another thing while reading through kerneltrap mailinglists: It seems most onboard so called IDE raid cards are actually just fancy labelled IDE adapters where the windows driver is nothing but a software raid.
[Rick] Yes. Avoid. Use the md driver, instead.
So raid with IDE raid cards are still software raid unless its the (rather expensive?) real thing.
[Rick] Yes. "Real thing" basically means 3Ware Escalade 850x or Adaptec AAR 24x0.
Promise was explicitly mentioned.
What I'm not sure is if this is for all raid modes or just raid 5.
[Rick] Please see my file. You can sometimes get RAID0 disk-spanning using a BIOS trick.
Simply doubling the writes for a raid 1 can't be that demanding in card logic after all.... Does anybody know more on this? Has anybody used IDE raid controllers? How do they behave?
[Rick] 3. Indeed, gigabit ethernet will strain practically any system, even with jumbo MTU sizes set, and even with a suitable switch and other associated equipment.
So that's true -- but then I've no immediate use for a Gbit ethernet now but I want to keep the option open. I think I'll watch the AMD64 scene for some more time, especially if hardware drivers for something like TV cards will run with 64bit OS.
If all this bleeding edge driver location stuff is getting sufficiently annoying I might even grab some PIII off ebay and just plug my existing SCSI chains in (and swap some 9GB drives in for the 2 and 4.5 one)
Still -- specs on that AMD64 sound nice.
[Rick] Yes, I certainly am with you, there! But I want to jump onto the 2nd generation of Opterons, not the first.
There might be a fair bit of video processing in the near future (as well as the use as home media center with timeshifting and all that stuff) so there will be incoming video streams on the PCI as well.
[Robos] we use a 400Mhz Celi for the (software) raid 5 and it works like a charm. Sometimes my flat-mate and I both stream stuff from there and no clogging or anything. Regarding timeshift: if you use dvb you can easily capture the stuff since it's already in mpeg2 format. No processor usage then. And any recent platter should be able to both record and play mpeg2 streams simultaneously.
I am toying with the idea of buying an athlon 3000+ soon: no intel (they have a too high profit margin for my liking), very performant, rather cheap, 64bit (for what it's worth) which linux (debian IIRC) supports rather good. But stay away from the vobis offering, that sucks
That AMD3000+ is intriguing -- what a small c't article can do to ones minds
[Robos] There was one before, like 5 numbers or so, which dealt with 64bit again 32 bit. You might want to read that before too.
From Ben Okopnik
Answered By: Thomas Adam, Kapil Hari Paranjape
So, I've installed swsusp - two patches over the kernel, flip a couple of switches in the kernel config, and "echo > /proc/swsusp/activate". Hurrah! Cute little screen with a progress bar, etc., and off goes the laptop. Now comes the interesting part: when I hit the power button, I see the Acer splash screen, the LILO message, my cute little Tux, and - the usual kernel load messages flip over to the 'restore the suspend' screen with its progress bar... until it gets to the point where it says "Freezing processes: Freezing remaining tasks." Half a second later, it restarts the restore process, until it gets to the above message... and so on, and so on, and so on. PLOKTA didn't do anything - until the next cycle, where the kernel crashed with a panic (!). I ended up having to reboot using another kernel - lucky that I always have at least two, since I don't trust the newly-compiled ones to always work!
There's no info about this on the Web, at least according to Google; just two hits on the relevant line in the code. I'll have to send some email to the "swsusp" list and see if they can help.
[Thomas] How are you actually suspending it though, Ben? I suspend this way:
echo 1 > /proc/acpi/sleep
Huh? Then you're not using "swsusp"'s hibernation feature, but the ACPI "sleep" instead. It's ">/proc/swsusp/activate", as shown in the swsusp HOWTO. Incidentally, the ACPI "sleep" doesn't work for me, whether 'echo 1' or 'echo 4'.
Yes, you are quite right, the two are different. I really am not sure what to suggest other than ask on the swsusp ML. I am not trying to fob you off or anything, but this kind of thing is quite hard to keep track of... i.e. we don't know what is happening until we reboot, etc....
grrr
Heck, Thomas, I certainly wasn't taking it badly. If the issue is new enough that I can't find anything on Google, then it's off to the developers I go; no one else is going to know anything.
Meanwhile, I've got to juggle the flight school appointment here with my trip to the Polynesian Culture Center (http://www.polynesia.com); I'll probably talk to you all next when I'm back in St. Augustine. Cheers!
[Kapil] There is a nice "comparison table" by Nigel Cunningham regarding the swsusp patches available at swsusp.sf.net.
Basically, there are three ways to get suspend-to-disk working for 2.6 kernels. There are two methods that are in the default kernel tarball. There is also a "swsusp2" patch that can be downloaded from the swsusp.sf.net site.
What is not mentioned (and not configured into the Kconfig scripts) is that the current versions appear to depend on the "module unloading" fature that is optional with the 2.6 kernel.
Secondly, while it is true that
echo -n disk > /sys/power/state
or
echo 4 > /proc/acpi/sleep
or
echo 1 > /proc/kernel/suspend/activate or whatever
should suspend-to-disk, it seems to be much smoother to use the "hibernate" script that can be downloaded as part of the tarball at swsusp.sf.net.
Finally, my own experience with the 1.0.3 patch for 2.4 is that it works quite well. I haven't had as much stability with the 2.6 suspend versions. (Works only while testing but fails when actually required .
Eventually, I'll get around to posting my questions to the swsusp site once I have all the variables sorted out.
[Heather] Hey gentle readers! If you know more on this topic please feel free to drop us a note - inquiring minds want working laptops
From Ben Okopnik
Answered By: Robos, Ben, Karl-Heinz Herrman, Jason Creighton, Thomas Adam
X11 and acer laptops ~~~~~~~~~~~~~~~~~~~~
This is by far the longest thread of the month with over fifty replies and many that I have deferred. I have tried to split it into sections where appropriate.... enjoy! -- Thomas Adam
I've got this brand-new Acer Aspire 2003LMi laptop - slick, sleek, and faster than a greased weasel on dexedrine. However... it doesn't seem to be too Linux-friendly. (( Sorta. Here are the details:
I can boot LNX-BBC, and even run X. So far, so good - but this is a
really high-res display with lots of fancy goodies. Besides, although
I remember LNX-BBC as having some sort of an HD install procedure, it's
really not what I want to install here.
I can boot Knoppix. Well, more or less, with a lot of emphasis on "less". It comes up, gets through the KDE stuff... and freezes about 10
[Robos] Try disabling acpi. In my laptop, if I move the mouse and kde wants to look what the battery status is the mouse jumps all over the place. Maybe yours is worse. append acpi=off should do the trick.
[Ben] Interestingly enough, ACPI is what works here, while APM doesn't. I've had to install the entire ACPI kit'n'kaboodle (kernel recompilation, userland stuff) and remove APM to make it work. Don't have suspend going yet, but all the other power management stuff works.
[Thomas] ACPI overrides APM in its functions and callings. On my laptop it works rather well, and coupled with swsusp is a dream to suspend to disk.
To get swsusp going, I cannot do better than to recommend the patch for it from:
http://swsusp.sf.net
A kernel compilation is again going to have to ensue, but afterwards, you can do cool things like:
sudo echo 4 > /proc/acpi/sleep
which will dump everything onto your swapspace.
That does look pretty cool... however, I'm not really all that wild about having to maintain a stable of patches. I might do it anyway - I really want that "suspend to swap" feature - but it just seems like an annoying thing to have to remember in addition to the kernel upgrades.
seconds after I start moving the mouse. "knoppix 2" (console only) works fine. "fbdev", "vesa", etc... none of the servers that I've tried for X work in any reasonable way.
[Robos] What's the graphics-card?
Radeon Mobility 9200 (9M+). New enough that "pci.ids" in the 2.6 kernel (I had to upgrade - just moving the mouse (Synaptics touchpad) would completely lock the machine with 2.4.22!) doesn't have it yet, although the folks that actually maintain "pci.ids" already have it. I'd tried, several times, to use the "experimental" xfree86 server, which supposedly can handle it via the "ati" driver, but no luck - so I'm stuck with the proprietary ATI driver. However, I'm a lot happier with that than Dell's nVidia; at least I know that in a year at most, ATI will release the interface.
[Thomas] Yeah, the experimental xserver-xfree86 package is the one you're going to have to use for this. I apt-get'ed it the other day.
There are four partitions on the new machine's HD: Wind0ws C:, Wind0ws D:, "suspend", and... ready for this?... Linux. I kid you not. This gadget has what's called "EFI", which I just happened to read about the day before I bought it by pure coincidence (I had no idea that this machine would have it) - it's a BIOS "replacement" that allows much easier pre-boot programming. What these folks have done, clever buggers that they are, is create an app - two of them, really - called "Arcade" (plays VCDs/DVDs, shows your picture repositories, etc. - sound familiar? Think "MoviX", etc.) and put a button on the front of the laptop. When the 'top is _off,_ pushing this button fires up "Arcade" without starting Wind0ws. When it's on, it fires up a Wind0ws app that looks exactly the same. Is this cool, or what? The way I found out
[Robos] This sounds cool. I heard about it but quickly forgot again.
that it was Linux is by running LNX-BBC and looking at the partitions. When I saw "/bin", "/boot", "/etc" and so on, I thought I was having a brain cramp... took me a bit to figure it out. Aside from that, though - obviously, FIPS won't work with this rig. What I need to do is shrink that first partition (Wind0ws) and put Linux into the newly-created second partition (I don't want to just blow away Wind0ws - there are a few gadgets here, such as a built-in SD/SmartMedia/etc. card reader that are going to take some research
[Robos] I'd say they connected this to the internal usb plug and so this should be easy to get working: enable "probe all LUNs" in the scsi section and then it should show up with scsi-emulation on. This is at least how my 6in1 card reader shows up when I plug it in (is external)
[Ben] before I can use them under Linux.) I wouldn't mind reinstalling Wind0ws after tweaking partitions - but all I have are "recovery" CDs (which will, presumably, restore the system to exactly the way it is now.)
[Robos] partimaged? Have some place to put the image? And I have had the experience with my backup-partition that when I did run it (when you activate the playing-back mode in the bios) it started dos and ran the win installer - LOL. But this way I was able to keep my partition the way they were and have a c: which is still fat32.
Hmmm... I just thought of something. I could blow it all away, install Linux - I'd have at least console-mode stuff, which would be sucky but workable, and copy everything over from the old machine. Then, when I come back, I could buy an external HD for backup, copy it all off, reinstall Wind0ws, and be abck to the same status. Hm. Frankly, this sounds really last-ditch. Worst of all, the new 'top has no serial port, so I wouldn't be able to use my cell phone to connect to the Net.
[Robos] Does your cell-phone have irda? Believe me, this is rather easy (if your acer has irda too and this is supported). I was surfing in my holidays back in the spring with my siemens ME45 lying beside my thinkpad r31. Connectivity!
[Ben] I did too. Didn't work for me.
[Thomas] The first thing I could be inclined you did, Ben, is provide your "lspci" output, along with "lspci -n". What you should probably ensure is that your PCI entry is valid for the detected card.
On my laptop, I have:
00:02.0 VGA compatible controller: Intel Corp. 82852/855GM Integrated Graphics Device (rev 01)
Re-run:
dpkg-reconfigure -plow xserver-xfree86
and when prompted for your PCI entry add
PCI:nn:nn:nn
where nn is your numbers from the correct entry for your card, but they HAVE to be in decimal format. lspci gives you them in hexadecimal and do you have to convert them. Using my example then...
PCI:00:02:00
I want you to try VESA this time, Ben. See if that makes a difference.
However without knowing much more information about your card specifically there is not much more I can do You should ensure that you have AGP support compiled into your kernel at least.
The thread then changes slightly, but we're still on similar lines -- Thomas Adam
OK, so here's the follow-up: since Thomas prodded me, and I have a weakness for unrestrained bouts of geekery anyway, I gave it one more try... and got some good results (didn't have much to do with configuring X, though. Sorry, pal.) I even got my framebuffer stuff working, including Tux at boot time - yay!
Oh well, didn't hurt -- Thomas Adam
I installed the 2.6.0-test11 kernel so I could do the "swsusp" patch - haven't yet, the network here at the hotel died for an hour or so and just came back up - and compiled it (I think this is where the framebuffer stuff came from; I hadn't changed anything in that section.) Previously, though, I got a very thin lead from a discussion I saw where somebody recommended turning off AGP in the kernel config, since the ATI driver does its own, followed by someone else recommending trying it both ways - according to them, some ATI cards will only work with ATI's AGP while some others work only with in-kernel AGP. Actually, that didn't help anything, but while twiddling this, and making corresponding tweaks in the ATI "fglrx" driver configuration, I got an error from trying to load "fglrx" - 'The fglrx module must be loaded before any other DRM driver!' From there, it was a short trip back to the kernel, compiling the ATI Radeon DRM as a _module,_ then loading "fglrx" followed by "radeon" in /etc/modules, and - /voila/! Well, partially.
I now have 1280x800 video with 24 bit color (still not fantastic as compared to what the card/screen can do, but certainly better.) According to the messages in the X log, this is the best that the ATI driver can do - which is pretty sad, but what can you do. Hopefully, they'll make it better as time goes on.
"swsusp" and the card reader are next on the list...
(For those of you who think I'm blowing my time in Hawaii on this stuff, wrong. I'm getting up at 4:30 a.m. local time - which is 9:30 a.m. by my internal clock, which I'm lucky enough to have decent control over - and banging away on this until it's time to go to work. Ooops - it's that time now!)
[Thomas] Odd. Did you:
cp /boot/config-$(uname -r) /usr/src/linux/.config
before you compiled the kernel?
If you try doing this from 2.4.X -> 2.6.X it will not work. In any case, I always like to build kernels using a fresh config file no matter what I last used.
[Ben] Not at all. My first experience with the 2.6 kernel was this past week, and I was greatly tickled by the fact that it automatically takes the previous .config and (obviously) does a bit of intelligent parsing. You can even (and this is what I did) do
make oldconfig
after installing the new kernel, and it will load up the old config file and only ask you about the differences (there weren't any between 2.6.0 and 2.6.0-test11.)
Well, at this point, I've managed to take a largish step back. :/ "swsusp" docs say that you should have at least $MEMSIZE * 1.3 in swap, so I blew off the empty partition that I had following swap, deleted both of them, resized swap and created a partition out of what remained... and now, when I boot, "fsck" falls down with a loud "THUD" and cries that it found an error and I should re-run it as root without "-a" or "-p". When I do so, it tells me I have a mismatch between what the BIOS and the partition info say (about 2,000 sectors) and requires me to press 'Enter' 6,000 times (it generates 3 error messages/prompts per sector.) No, you cannot pipe the output of 'yes' to it. Laying a weight on the 'Enter' key (hi-tech solutions is us!) gets it done in about five minutes... after which it proceeds to repeat exactly what it did before. However, in this case, "Hit Ctrl-D to continue normal startup" actually does work, so here I am. [sigh]
[Thomas] Hmm, did you use cfdisk for this? It might be that you have to have a new map file in your /boot partition. Try running lilo again to see if that'll kick it? It might be worth just disabling swap for the time being and manually running:
swapon /dev/xxx
later on.
I used cfdisk. Deleted them both, made two new ones using the space. However, it's fixed already. Deleted them again, rebooted with them still as free space (that way I knew that "fsck" couldn't complain about partition mismatches there), then recreated them on the next boot. Whew.
[Robos] There have been sone synaptic touchpad fixes gone into 2.6.1 according to changelog.
[Ben] Not enough of them - tapping the touchpad, which has always worked, got borked in the process, and "gpm" doesn't work with the "synaptic" driver in the kernel (TONS of spurious messages.) Fortunately, I found patches for both of them. Tapping still doesn't work in the console, but it's not a huge issue. Also, the touchpad buttons are arranged like this:
_______________________ | | | | | | | Touchpad | | | | | | | | | ----------------------- | _|_ | | / \ | | Button | X | Button | | 1 \___/ 2 | | | | -----------------------
where 'X' is the four-directional 'scroll' button... which makes it damn near impossible to click 1+2 to paste. There's yet another patch that allows 'up' on the scroll button to be used as button 3, but I haven't managed to make it work yet.
Other than that, I'm actually doing reasonably OK with it for the amount of time I've spent tweaking it:
Working: 1024x768 video Sound Touchpad (except the above issues) NIC USB2.0 ACPI (no suspend yet, though) Untested but everything loads OK: Wireless networking Bluetooth interface CD-RW/DVD-RW Firewire Parallel port Video out FIR Not working yet: 4-in-1 card reader Better video Framebuffer Modem (Lucent winmodem, bleh :( There's probably a binary driver somewhere, but I think I'll get a PCMCIA modem.) Five-way "media control" keys
[K.-H] Dell Inspirons (and maybe others) had 1600x1200/15" for some years now at least starting with the Insp. 8000 on which I'm typing. There were Insp. 5000 with high res 15" screens as well. The current 8600 or higher have an even higher resolution on the top models. As you sit rather close to a lapscreen I like the display. I had to switch some fonts to a slightly larger one, but the displays are very sharp at the native resolution and a 10pt full A4 page in gv (antialias on) is quite readable.
Currently I'm thinking about a desk TFT -- and am rather unimpressed as you almost can't get more then 1024x768 up to 17", then 19" have 1280xWhatever. Admittedly that screen is further back from the keyboard, but why is no company offering any TFT with a higher res. They are possible after all.
The Inspirons use either nvidia Geforce 2 (or up in later models) or ATI Cards -- which won't help you Ben. I hope you figure out how to get into native resolution. For most others XFree4 seems to calculate good modelines by itself (I've not specified any). As yours is slightly weird maybe you need to specify one, keep looking on google.
Then there's the BIOS to look at... -- Heather Stern
There are four partitions on the new machine's HD: Wind0ws C:, Wind0ws D:, "suspend", and... ready for this?... Linux. I kid you not. This gadget has what's called "EFI", which I just happened to read about the day before I bought it by pure coincidence (I had no idea that this machine would have it) - it's a BIOS "replacement" that allows much easier pre-boot programming. What these folks have done, clever buggers that they are, is create an app - two of them, really - called "Arcade" (plays VCDs/DVDs, shows your picture repositories, etc. - sound familiar? Think "MoviX", etc.) and put a button on the front of the laptop. When the 'top is _off,_ pushing this button fires up "Arcade" without starting Wind0ws. When it's on, it fires up a Wind0ws app that looks exactly the same. Is this cool, or what? The way I found out that it was Linux is by running LNX-BBC and looking at the partitions. When I saw "/bin", "/boot", "/etc" and so on, I thought I was having a brain cramp... took me a bit to figure it out.
[Jason] Okay, so there is actually some version of Linux installed on the laptop? That is really interesting. If you have time (Ha ha!), I'd really like it if you could post some details sometime.
I don't have the HD installed (although I have it with me), but what I remember of it is this: it's a minimal (sorta like a "chroot" jail) system with a large tarball in "/". Looking inside the tarball revealed a more complete system; presumably, it works something like the LNX-BBC system (Oy, Heather!) by using the basic system to fire up, then decompress the tarball into memory and "pivot_root" (I'm guessing here.)
[Jason] BTW, when I first heard about EFI, my first thought was "Cool!". My second thought was "Somebody is going to code up a Tetris clone for this thing". Really, what could be more fun than playing tetris without having to boot up an OS? Actually, I expect people to more or less treat it like another OS, because, AFAIK, that's basically what it is. So expect all manner of useless programs.
I've explored it a little further, and it seems to be some Chinese-assembled version of Linux; fires up, opens up a tar archive, chroots into it, and runs a rather flashy Linux app (a front-end for lots of neat-o music/DVD/CD/etc. utils - mostly based on "mplayer", AFAICT.)
[Robos] Well, you should then write the mplayer folks a nice email telling them that. Quite recently they even started a new ml called mplayer-legal... Or do you think you can get the sources somewhere?
[Ben] It's quite cute, with a bunch of "lilo.conf.X" entries in /etc - obviously in case the partition ends up being anything other than hda4 which is where it is by default. Cuter yet, there's no alternate VT you can flip to - but there is a getty running on a serial port, and "/etc/passwd" doesn't have an entry for "root" but does contain one for "ava:0:0:..." Ain't that preeeecious?
Heck, I might just copy the thing and put in on a CD for a stand-alone movie/slideshow/etc. player.
I've actually considered doing that [a howto], on the principle that if I had that much trouble, other people surely will as well. However, I've held off so far because I'm not quite finished yet; I've zapped the original HD back into the Acer, and am now trying to split hda1 into hda1 and hda5, for Wind*ws on 1 (10GB or so) and Linux on 5 (40GB). However, the DVD+R media that I bought for backing up hda1 turns out to be incompatible with the drive (which does only DVD-R and down), so I have to go back to the store.... all this in between yoga classes, a flight medical (I now have my class III flying license), actual flying, chanty singing with a reenactment group, trying to somehow get Net access going (I've sorta got it - by buying a new cellphone), patching a leak that my dinghy has mysteriously developed, dealing with a couple of clients who have picked this time to play some highly shitful games with paying me (until I threatened legal action), fixing a recalcitrant propane heater in my aft cabin, and handling the initial stages of what looks like a local romantic involvement.
Actually, Ben did just that. The URL for which is here:
http://okopnik.freeshell.org/acer -- Thomas Adam
By Ray Ingles
"There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies, and the other is to make it so complicated that there are no obvious deficiencies." - C.A.R. Hoare
"Sure I'm paranoid, but am I paranoid ENOUGH?" - Unknown
System administrators frequently want to be able to work on the machines they run even when they are far away from them. There are secure tools that allow full remote shell access, like ssh and lsh, but due to their complexity they have suffered critical exploits from time to time. In addition, their overhead can be excessive for some purposes. Fortunately, other options are available that can be used alone or can be combined with remote shells to create a more secure overall system.
Maybe the pager has just gone off when you're home in bed, and the boss wants you to fix the broken database now. Or perhaps you're out for lunch and someone calls to tell you the mailserver has been cracked and is currently spamming the world, and you need to bring it down fast. Possibly you've checked and your Web server has wedged itself and needs to be restarted. Or suppose you're just on vacation and find you want to update your home Web site with some new photos. In all these cases, you'd like to do something to the machine over the Internet without having to actually sit in front of it - things you don't want just anybody to be able to do.
Tools like ssh and lsh are great for allowing secure remote access to your system. They offer essentially full, flexible remote control of a machine, in an encrypted and authenticated manner. But they are complex pieces of software; there's no way to do what they do without being complex. And with complexity comes bugs. SSH and lsh, and related tools like Webmin, have all had serious flaws that would allow an attacker to get full control over your system. Leaving them available all the time is a risk - sometimes it's necessary, but it's still a risk. And in some cases, you'd like to be able to tell the machine to do something, but it's not even attached to the network on a regular basis.
It would be nice to enable remote shell access only when necessary. And perhaps (for something like shutting down a mail server) you don't even need a full shell, just a way to fire off a script remotely. Of course, the problem then becomes, how do you know that the alternative software is any more secure than ssh itself? Various people have worked on this problem in the past, and several potential solutions are available, ranging from the simple and venerable to the new and exotic.
Xringd uses a modem to control a machine remotely. Mail filters can be used to trigger actions based on special messages. Some solutions (like 'port knocking' and 'Net::Pcap') use the network, but without requiring even a single open port. Lando runs commands over a network, using username and password. Most recently, a program specifically for secure remote execution called Ostiary has been developed.
The eXtended Ring Daemon, or "Xringd", uses a modem to monitor rings on a phone line. It counts the number of rings, and the time between them. If a 'sequence' matches one of the ones that it has been set up to detect, Xringd will run an associated command.
This is very nice from a security perspective. Since it uses no network connection at all, it's entirely immune to network attacks like buffer overflows. It can be used even when a network connection is unavailable (it's often used to cause a computer to initiate a dialup connection). The only 'client' you need is a phone. If you use it to start up ssh on demand, then the attacker needs to know the right phone number and the right ring pattern - it's quite hard to sniff that kind of thing remotely. It's also highly resistant to a man in the middle attack. (If you have to worry about someone rerouting your phone calls, you're in more trouble than Xringd can save you from.)
There are some practical issues that may make this unattractive in some circumstances. You need a modem and a telephone line to the server. (Fortunately, you don't need a fast modem at all; even a 1200 baud one will do nicely, but some servers are not placed close to a telephone jack.) Also, things like answering machines or voicemail (or even other people answering the telephone) can interfere with Xringd. If you give the server a dedicated line, you can avoid these problems, but that can be costly.
Finally, note that the rings you hear when making a call are not necessarily synchronized with the ring signals actually sent to the telephone. In most circumstances, they are close enough, but reliability can be an issue at times.
Most of the mail filtering programs have a way to invoke scripts when mail matching a pattern is received (in the simplest case, mail to a particular address). Assuming the server is running an SMTP daemon, this can be a nice way of triggering actions remotely. Technically, one could even send a shell script to be run, and have it e-mail the results back to you, giving you the equivalent of a very slow remote shell. The only client needed is an e-mail program, or even a webmail account.
The first problem is that if the box you want to talk to doesn't accept e-mail, this obviously won't work. (Adding an entire mail server, with the attendant risks of bugs, spam load, etc., just for remote execution doesn't make a lot of sense.) Some machines only periodically collect e-mail from a primary server, so there can be a substantial delay between when a command is sent and when it is acted upon.
Furthermore, if you don't encrypt the traffic in some way (or at least sign it with PGP), then anyone sniffing traffic between you and your server may be able to take advantage of the same channel to do mischief, or perform a man-in-the-middle-attack. (E-mail traffic is notoriously easy to falsify; hence the avalanche of spam these days.)
CVTSA, or "ClairVoyanT SysAdmin", is a system designed specifically for running commands through e-mail. It has some support for using passwords, but does not (currently) encrypt them in transit, so a sniffer could capture them and use them again.
Of course, if the only things you want to do with this type of system are emergency shutdowns and other such (hopefully rare) crisis management, then even an unencrypted channel might work. However, you'll need to change the 'magic trigger pattern' each time after you use it, or you take the risk that an attacker might capture it and 'replay' it at an inconvenient time.
With port knocking, a daemon monitors firewall logs, looking for particular sequences of connection attempts to particular (closed) ports. When it sees a sequence it recognizes, it runs the associated command. This isn't terribly bandwidth efficient, but it has some nice properties. First, it's hard to tell if a server is listening for port knocks. Second (and most important), it's awfully hard to crack a closed port. (Linksys routers have had a simple version of this for a while, BTW, that they call port triggering.)
However, a clever attacker with a sniffer could notice this traffic, and duplicate it for their own use. More complicated encodings could express something like a PGP signature (indeed, in theory one could create an entire network protocol based on port knocks), but things rapidly become difficult to work with. As with 'mail filtering' solutions, one can either use it sparingly in emergencies, or move to real cryptography.
It's also important to realize that this system is critically dependent on the probe packets actually being delivered, and delivered in the order that they were sent. This is not guaranteed on the Internet. What's more, depending on where you're at (e.g., an Internet cafe or behind a business firewall), you might not be allowed to connect out to arbitrary ports. The more complex you make the 'knocks', the less reliable the system will be.
Also, notice that at least one entire IP packet (28 bytes or so minimum) is used to transmit roughly one bit of information. In terms of network efficiency, it's almost hideous. For a simple 'open up ssh' message, it's not a consideration, but actually adding cryptographic security to this system could use up a decent chunk of the available bandwidth.
Finally, this increases the CPU load for each entry in the firewall log. Depending on how detailed the logs are, and how fast and busy the network is, this can be a significant drain on resources.
Another interesting approach is to use Net::Pcap or other network capturing software to look for specific packets on the network (e.g., DNS requests) and examine them for particular data (e.g., a particular address). If found, it can enable ssh temporarily, or perform other actions.
One potential benefit of this approach is that a computer doesn't have to have an address on a network in order to monitor traffic on that network. You can set the card to 'promiscuous mode' and examine all the traffic on the wire. (It's very hard to hack a machine you don't even know is there.) Once the 'trigger' is spotted, the sniffer can use other means (a separate network, a serial link, even Xringd) to open up SSH on a target machine. Of course, you can also simply run the sniffer directly on the target.
Again, a clever attacker with their own sniffer may be able to detect the unusual activity and correlate it. To make this system truly secure, you would need more complex encoding/encryption of the 'trigger' traffic.
Additionally, the CPU load for this solution can be even worse than for 'port knocking' systems. A 'port knocking' daemon monitors firewall logs, which can have variable levels of detail. By necessity, a 'sniffer' solution must examine every packet on the network segment, which can be a substantial task for a busy gigabit line.
Lando allows a user to run a preconfigured set of commands remotely, using passwords, and even allowing the user to supply arguments to them. While it currently has only a Windows client, and passwords are sent in the clear (making it suitable only for use on a trusted local network, or perhaps on a VPN), it can be very useful for, e.g. operating a local firewall box without going to the trouble of logging in.
All of the above solutions have their advantages, but each has some practical issues that can make them unsuitable for particular applications. Ostiary was designed to be a secure alternative that uses minimal resources. It tackles this problem with what might be termed "aggressive simplicity". It does require an active connection to the network (unlike Xringd and sniffing), but allows for much better default security with very low CPU, RAM, disk, and network bandwidth requirements.
An Ostiary server has one open port that it listens on. When someone connects, the server sends a random fixed length 'salt' message 16 bytes in size - the size of an MD5 hash. It then waits (with a timeout) for a reply from the client. It reads (at most) 16 bytes of reply, and closes the connection.
Ostiary has a list of commands to run, with associated passwords. It runs through the list, and hashes these passwords with the 'salt' it sent to the client. If one of these hashes matches the reply from the client, the associated command is run. (One final touch is that a record is kept of connections, and clients with too many failed attempts are 'locked out', and all subsequent communication from them is ignored.)
A detailed security analysis is available, but a few things about this system should be clear. With a protocol this simple, the chances for dangerous bugs are drastically reduced. Using fixed-length messages essentially eliminates the chances of a buffer overflow or other memory error. (Indeed, Ostiary does no dynamic memory allocation of any kind - everything is stored in static, fixed-size data structures.) Replay and man-in-the-middle attacks are also effectively useless. Ostiary limits how fast it accepts connections, enforcing low CPU and network usage. (The first production Ostiary server was a 16MHz 68030 machine.) Client requirements are even lower: Clients are available for Palm Pilots and even Windows.
Unlike a procmail-based solution, where you can put arbitrary commands (with arguments) in the message, Ostiary can only run the fixed set of commands you have preconfigured. The only argument it supplies to the commands is the IP address of the client that initiated the command. It requires an active network connection (unlike Xringd) and an open port (unlike port knocking or sniffing), which may entail configuring a firewall to open a new port. (Although one could run Ostiary on, say, port 22, and upon receipt of the correct command, it could terminate itself and spawn sshd...)
Since Ostiary uses TCP, it is as reliable as the network it uses to communicate. Problems with miscounted phone rings (a la Xringd) or randomly dropped packets (a la port knocking) are not a concern.
The following table summarizes the pros and cons of the various systems outlined above. "Replay" and "Man-in-the-middle" indicate if the default system is vulnerable to the corresponding attacks. "Command arguments" indicates if the system can run arbitrary commands with arguments. "CPU load" indicates that CPU time can be a significant consideration. "Special client" indicates that a specific client program is needed to work with that system.
System | Xringd | Mail filter | Port knocking | Sniffers | Lando | Ostiary |
Network Required? | Yes | Yes | Yes | Yes | Yes | |
Port Required? | Yes | Yes | Yes | |||
Modem Required? | Yes | |||||
Replay? | Yes | Yes | Yes | Yes | ||
Man-in-the-middle? | Yes | Yes | Yes | Yes | ||
Command arguments? | Yes | Yes | ||||
CPU load? | Sometimes | Yes | Yes | |||
Special client? | Sometimes | Sometimes | Yes | Yes |
None of these approaches is right for everyone. But all of them can be used to make attacks at least more inconvenient, and in many cases far more difficult. Remember, though, to analyze their pros and cons relative to your specific situation. Also remember that true security is a process, not a goal - you can never just install some software and be done thinking about it.
Ray Ingles has been involved with Linux since 1995. In
addition to being an active member of the
Metro Detroit Linux User's Group,
he has made minor contributions to the UPS HOWTO and the Linux
Joystick Driver.
By Bill Lovett
Where do you go to find out what's on TV? The usual suspects might include a newspaper, a recent issue of TV Guide magazine, a favorite Web site, or your nearest TiVo, ReplayTV, or other PVR. But don't forget to add Linux to the top of that list. You can let the machine do the dirty work and bring the listings to you. XMLTV, a short bash script, and a cron job are all you need to get started.
First things first: getting the program installed. XMLTV is a suite of Perl scripts and can be downloaded from membled.com/work/apps/xmltv. There are releases for Unix-like and Windows environments, but, for obvious reasons, we'll focus on the former. If you're installing from source, it's the usual routine:
% perl Makefile.PL % make % make test % make install
If you're on Debian, it's all just an apt-get away
(apt-cache search xmltv
). Links to packages for OS X, Red Hat 8, and Red Hat 9 are
available from the project's homepage.
Before XMLTV can be useful, it needs to know where in
the world you are. XMLTV is international it can fetch TV listings for
Canada and the United States, the United Kingdom, Austria and Germany, New
Zealand, Finland, Italy, Spain, the Netherlands, Denmark, and Hungary. (Belgium
and France are in the works.) The scripts that collect listings for a
particular country are referred to as grabbers, and you'll find them on the
command line under tv_grab_*
. We'll use the U.S. grabber,
tv_grab_na
.
When you first run the grabber, do so with the --configure
option. This
starts a question-and-answer session in which you and the grabber get a little
bit better acquainted, as far as your Zip code, TV service provider, and
channels you want to ignore are concerned. The results of the script are written to
~/.xmltv/tv_grab_na.conf
, and can be easily edited by hand.
At this point, XMLTV is ready to do your bidding. Do a man tv_grab_na
to learn
about all the available options. For now, just two will suffice:
% tv_grab_na --days 1 --output /tmp/tv.xmlThis tells the grabber to get one day's worth of listings, and save them out to
/tmp/tv.xml
.
XMLTV's file format doesn't quite make for friendly reading, unless you
enjoy reading raw markup. A few more scripts from the suite can fix that. tv_sort
sorts the contents of an xmltv file date.
tv_grep
lets you weed out some of the obvious cruft in the
listings. Here's how I run it:
% tv_sort --output /tmp/tv_sorted.xml /tmp/tv.xml % tv_grep --output /tmp/tv_grepped.xml --ignore-case --not --category Children \ --not --category Sports --not --title "Paid Programming" \ --not --title "Local Origination" \ --on-after now /tmp/tv_sorted.xmlThe commands above sort the original file and then discard anything categorized as "Children" or "Sports", and anything with "Paid Programming" (infomercials) or "Local Origination" (public access) in the title. Also, we're discarding everything that aired before the script ran.
At this point, we've still got an XML file. Converters to the rescue!
tv_to_text
is one of the tools that can help us go from XML to something else.
(Other possibilities include LaTeX, HTML and PDF. Check the readme to see what's
currently available.) After running something like this:
% tv_to_text --output /tmp/tv.txt /tmp/tv_grepped.xmlWe get output like this:
21:00--21:30 Spy School 38 21:00--21:30 Designing for the Sexes // European Kitchen 64 21:00--21:30 Chappelle's Show 67 21:00--21:30 The Real World // Las Vegas 71 21:00--22:00 Law & Order: Special Victims Unit // Guilt 44 21:00--22:00 Wild Card // Auntie Venom 45 21:00--22:00 Cold Case Files // The Accidental Killer; Little Sister Lost 57 21:00--22:00 America's Most Wanted: America Fights Back // Top Ten Most Wanted Fugitives 5 21:00--22:00 The FBI Files // The Price of Greed 60 21:00--22:00 Trading Spaces // Nashville: Murphywood Crossing 61 21:00--22:00 Great Chowder Cook-Off 63 21:00--22:00 Ends of the Earth // Secrets of the Holy Land 65 21:00--22:00 The E! True Hollywood Story // The Hilton Sisters 68 ...Simple and no frills. Just what we need for the final step: e-mail delivery.
If we stopped at this point we'd have used several of XMLTV's abilities but
hardly anything else. We'd also be running low on convenience and automation.
Fortunately, we can wrap all the commands we've seen so far into a shell
script, and have it e-mail us the final results. mail
can take care
of, well, the mailing:
% mail -s "Today's TV listings from XMLTV" user@localhost < /tmp/tv.txtHere's what the full script looks like (text version of this listing):
#!/bin/sh # Grab today's listings: tv_grab_na --days 1 --output /tmp/tv.xml # Sort tv_sort --output /tmp/tv_sorted.xml /tmp/tv.xml # Grep tv_grep --output /tmp/tv_grepped.xml --ignore-case --not --category Children \ --not --category Sports --not --title "Paid Programming" \ --not --title "Local Origination" \ --on-after now /tmp/tv_sorted.xml # Convert To Text tv_to_text --output /tmp/tv.txt /tmp/tv_grepped.xml # Email mail -s "Today's TV listings from XMLTV" user@localhost < /tmp/tv.txtPut that in a cron job that runs once per day, and you've got TV listings with no outside advertising, and no channels or shows you know you aren't interested in.
More importantly, you've got a foundation to build on. What we've covered is just the beginning beyond the command-line scripts, a GUI client is also available. Of course, there are plenty more things you could do from the command line, such as:
tv_imdb
tv_split
Bill Lovett is a Web developer in New York City. He's one of those PHP/MySQL types. And he has this weird thing about running Linux on old machines that by all rights should have been trashed years ago. Read more about Bill and his Open Source projects at www.ilovett.com
By Pramode C.E.
Many of us make a living out of Linux - but, if somebody asks us why we are so crazy about it, one common answer would be `fun'. Playing with Linux is lots of fun - with the added benefit that, most of the time, you end up learning a lot. Recently, I happened to come across a nice book which tries to emphasize the `fun' aspect of Linux - it describes several small `projects' (a jukebox, a picture frame, etc.) that a moderately experienced Linux user may be able to implement on her own. One of the projects involved interfacing with a temperature-sensing element and putting up the temperature value on a Web page (or including it in your email signature - and any other crazy stuff which you can imagine!). The only trouble was that, in the part of the world where I live, walking up to an electronics store and asking for an integrated, 1-wire temperature-sensing element is more likely to yield a hard stare than anything else. Smart sensors that can be directly interfaced to the PC with the minimum of fuss are seldom available off-the-shelf - you will mostly have to `roll your own' - which adds to the fun and excitement. With a low-cost general purpose microcontroller like the PIC16F628, bits and pieces of cheap, commonly available electronics components, and LOTS of code, you can build many interesting `toys' and hook them up to your Linux machine - a really great learning experience for the hardware hacker who wants to learn Linux, or the Linux hacker who wants to learn a bit of hardware. This article describes how I went about building my temperature-sensing project - amateur Linux/hardware hackers might find some of the ideas useful when they start building things on their own.
This is the first step. Microchip PIC controllers are commonly available. If you are like me, working with a soldering iron for more than 10 minutes would drive you crazy - so you have to choose the right kind of PIC - the one that can be programmed with the simplest possible circuit (connected to the PC parallel port), preferably with a 5V supply. Look no further than the PIC16F628. This is a cool device that has lots of peripherals (except the ADC - but then, we can roll our own crude analog-to-digital converter with the comparator and pulse width modulation facilities offered by the PIC) and supports a `Low-Voltage Programming Mode'. I found a nice little circuit (the simplest circuit, and one that works perfectly, out of the dozens I have seen on the Net) designed by Jim Paris for a microcontroller programming laboratory at MIT. Here is the circuit:
[diagram]
I assembled the circuit on a breadboard for testing in a few minutes' time.
Jim Paris has designed a program (called `jimpic') for burning machine code onto the flash memory of the microcontroller. It is available for download from here. I wrote a simple assembly language program, converted it into machine code with the help of the `gpasm' assembler available as part of the GNU PIC Utilities Project and burned it onto the micro by running `jimpic' with the `-b' option.
A nice thing about the PIC is that, if you have some background in general microprocessor architecture and assembly language programming, you can become productive with it in just about one or two hours' time. The instruction set is very compact (35 instructions) and sufficient for most simple bit-twiddling tasks. The PIC16F628 packs a decent 224 bytes of data memory with 2K of program (code) memory. The peripherals include general-purpose digital I/O ports, three timers, two analog comparators, on-chip voltage reference module, Universal Synchronous-Asynchronous Receiver Transmitter (for serial communication), and Capture-Compare-PWM module. Special CPU features include a watchdog timer, brown-out detect circuitry, and an internal RC oscillator (so that you won't be needing an external crystal if you aren't too concerned about precise timing).
The general purpose data RAM begins at address 0x20 (the locations below this are Special Function Registers - basically memory mapped I/O ports, control registers etc.). Here is an elementary assembly language program, which simply stores the value 0 into the accumulator (the `W' register, in PIC terminology).
(Remove the .txt extension if you download the listing. It's there only to ensure browsers display it properly.)
We will now assemble the file:
gpasm -a inhx8m a.s
The result is an Intel hex format file, which can be given to `jimpic' for burning. Each line of the hex file contains a few bytes of machine code, the address at which the machine code is to be stored (in the flash memory of the microcontroller), some kind of checksum, and some other information. Here is the hex file generated by running `gpasm' over our assembly language program:
:020000000030CE :02400E00983FD9 :00000001FF
The first line of our program tells the assembler that machine code is to be generated for the PIC16F628. The second line includes a file (available with the `gputils' distribution) that contains lots of symbol definitions. The third line, a __CONFIG directive, tells the assembler what special features of the microcontroller (say, the Watchdog timer) should be enabled/disabled by writing bit patterns to a `magic' `configuration word' within the PIC; _WDT_OFF means we don't want the watchdog to be enabled, _INTRC_OSC_NOCLKOUT means we are going to use the internal oscillator to provide the timing signals necessary for program execution. You will have to refer to the 16F628 datasheet to know more about these configuration bits. The fourth line is the only proper assembly language instruction in the program - it moves the `literal' (constant) value 0 to the `W' register. Note that each line begins with a tab.
Here is a program that lights up an LED connected to the RB0 pin of the microcontroller:
PORTB is an eight-bit port - the direction of each pin (i.e., whether the pin is to act as input or output) is controlled by individual bits of the TRISB register - if a TRISB bit is set, the corresponding PORTB pin is input - otherwise it is output. The PIC has the concept of `banked' addresses, which is rather confusing to the beginner. (It's a headache even if you are an `experienced' developer.) You visualize `banks' of special function registers - the STATUS register is the same across all the banks while the TRISB register is available only in bank 1. You are by default in bank 0. To access TRISB, you have to `switch over' to bank 1. This is by setting the RP0 bit of the status register. (When you read microcontroller manuals, you will see that not only are the control registers given special names, even the individual bits are named. Header files available with the development kit for the microcontroller map these symbolic names to the numbers given in the manual, making the life of the assembly programmer a bit easier.) The `bsf' instruction (bit set f - `f' represents the fact that the number that comes as the operand for the instruction represents a memory address or a special function register and not a `literal') takes two operands - the first one being the address of a RAM location or a special function register, and the second, a bit number. The `movwf' instruction copies the contents of the `W' register to the memory location whose address is the operand of the instruction.
After assembling and burning the above program, we are ready to see it in action. The running circuit can be built in a jiffy - place +5V on the VDD pin of the PIC (pin 14), connect Vss (pin 5) to circuit ground, connect MCLR (pin 4) to +5V through a 2K resistor, connect the LED between RB0 and Gnd with a current limiting resistor of say 1K in series - and that's all. You should see the LED lighting up as soon as you apply power. Your next attempt will be to make the LED blink - for that you will have to read a little bit more about the PIC instruction set - the manual will come in handy at this juncture.
Here are some things that I have found handy while debugging:
The LM35 is a commonly available calibrated temperature sensor that converts temperature (in degrees Celsius) to voltage - each degree rise in temperature results in 10mV rise in output voltage. It's a three-pin device - Vcc, Gnd, and voltage output. You can get the datasheet from here. Say the current temperature is 23 degrees Celsius; the voltage output would be 230 millivolts.
The question is, how do you convert this voltage to a digital value? The easiest way would be to use a commodity analog-to-digital converter, and interface it to the printer port. Another solution would be to use a PIC with a built in ADC (say the 12F675). The third would be to use some of the peripherals available in the 16F628, write some code, and build a crude ADC of your own. As I had explored the first two options a lot in the past, I thought of trying out the third one.
Two peripheral features of the PIC are of interest to us here - one is the builtin PWM module (Pulse Width Modulation), which is capable of generating, in, hardware, a continuous stream of digital on-off pulses whose duty cycle can be varied simply by storing certain numbers in specific special function registers. Once the PWM module is initialized to generate a pulse train of a specific duty cycle, it will keep on doing so without any software intervention - our program can do something else.
The PIC is also equipped with two analog comparators, which can be configured in a variety of ways. Let's say we are using just one of the comparators. Two PORTA pins can be programmed to accept voltage levels and transmit them to the Vin+ and Vin- pins of the comparator. The comparator output is high if the Vin+ voltage is greater than the Vin- voltage, and low otherwise. The output can be made available on another PORTA pin, or it can be simply read from a particular bit of the Comparator Control Register, CMCON.
The figure shows a PWM pulse (off OV, on +5V) of period T being fed to an RC circuit (R*C >> T). If the on-off periods are equal, the output seen across the capacitor will be a constant DC level of magnitude 2.5V. Electrical engineering text books should give you the reason why it is so - or, if you are not very sure of the math involved (as I am), play with some R and C values until you get the desired effect. Now what if you feed a PWM pulse whose on-time is less than T/2? You will see that the output is again a DC level, but the magnitude has come down proportionately. What if you increase the on-time? Again, the output is a DC level, only thing is the magnitude has increased proportionately. Now you have a cool way to implement a DAC, a digital-to-analog converter. Say you want to generate a voltage of 0.449V. What if you program the PIC so as to generate a PWM pulse train of period 256 microseconds and on-time 128micro. The output voltage would be 5V*(128/256.0) = 2.5V. Now, what if the on-time is 23 micro seconds? The output is 5V*(23.0/256) = 0.449V. (I use Python to do these quick-and-dirty calculations. It's one of my favourite uses of this great language.) The on-time can be altered easily by writing some numbers to two registers, CCPR1L and CCP1CON. A pure digital way to generate analog voltage!
What has all this got to do with converting the LM35 sensor's analog voltage output to a numerical value? Well, a DAC, together with a comparator, builds up an ADC. How? Say the maximum and minimum temperatures at your place of residence can never go above/below 45 degree Celsius and 20 degree Celsius. So the sensor's output will always be between .45V and .2V (remember, 10mV per degree change in temperature). We start generating a PWM signal of period 256 microseconds. The RC-filtered output is fed to Vin+ of the comparator, and the sensor's output is fed to Vin-. Let's say the sensor output is .3V. If the PWM on-period is 23 microseconds, the filtered DC level would be 5*(23.0/256) = 0.449V. So, Vin+ is greater than Vin-, and the comparator output (as obtained from a bit of the CMCON register) is high. Now, we start bringing down the on-time. At a certain point, Vin- will go above Vin+, and the comparator output drops to zero. The magnitude of the on-time at this point is a true representation of the analog value of the sensor output. We communicate this number to a program running on the Linux box through a serial link. You can download the PIC assembly language program that does all these tricks:
Instead of performing a `linear' search from the high boundary down to the lower one, we can think of generating a voltage that lies in the middle of this range and comparing it with the sensor output. If the comparator says that the sensor output is higher, we can repeat the same procedure on the upper half. This is the classical `binary search' applied to solve a hardware problem! Horowitz and Hill, in their book The Art of Electronics, have oscilloscope traces of this binary search in action! Computer programmers should always show a good amount of skepticism towards code that claims to do binary search - the algorithm looks deceptively simple - but is in fact not very easy to implement correctly.
The PIC micro sends the temperature data it has gathered out through a port pin (RB2) in a serial manner - this port pin is directly connected to the receive pin of the PC serial port. What remains is to write a program that will read this data and process it in some manner. Even though the RS-232C serial communication standard defines an `on' voltage to be between -3 and -12V and an `off' to be between +3 and +12, I have been able to get satisfactory results using the 0 and 5V logic outputs from the PIC port pin - if it doesn't work out for you, you will have to place a device like the MAX232 between the PIC port pin and the PC serial port receive pin.
Let's look at the simplest way to interface an external circuit to the serial port. (We won't be sending any data out through the PC serial port - that would make the circuit a wee bit more complex.) Pin number 2 of the 9-pin PC serial port connector is the receive pin, 3 the transmit pin, and 5, Ground. Let's say the PIC is sending data out through its RB2 pin at 9600 bits per second, 8N1 (8 data bits, no parity, 1 stop bit) format. The UART that controls the PC serial port should be programmed for this particular baud rate and data format. This can be done by writing magic bit patterns to certain control registers. Once that is done, our program can keep on polling a bit of the UART status register to know whether a new data byte has arrived. Here is the code listing:
The program has two disadvantages. One, it is using low-level I/O calls, which, if they are to work properly, should be preceded by an iopl() call. Only the superuser can call iopl() successfully - so the program should run under root privilege. We are wasting CPU time when we keep polling for data in a tight loop; that's another big problem. Both are solved by not directly interacting with the hardware - we can make use of system calls to talk to the serial driver within the Linux kernel - which will do all the low level stuff needed to manage blocking, interrupt driven I/O.
The Python `termios' module can be used for doing serial comm at a higher level. Isaac Barona Martinez has written a simple wrapper over `termios'. It is called uspp and is available for download from here. Using this module, reading from the serial port is a breeze:
from uspp import * # COM1 is initialized at 9600 baud. The # default data format is 8N1 s = SerialPort("/dev/ttyS0", None, 9600) s.flush() # discard unread bytes print ord(s.read()) # s.read() returns a one-character # string. We convert it into its ascii # valueOnce you get this far, let your imagination run riot!
Thanks to Christopher Negus and Chuck Wolber for a really cool book!
Thanks to Jim Paris, Ariel Rodriguez and Sheldon Chan for the excellent `jimpic' hardware and software. As I had mentioned earlier in this article, I find it to be the easiest way to get started with PIC programming under Linux. Thanks to Isaac Barona Martinez for uspp, which simplifies serial programming a lot.
There are two excellent documents that describe serial programming under Linux. One is the Serial Programming HOWTO. The other is Serial Programming guide for POSIX operating Systems. The Microchip home page contains lots of application notes, reading which might give you ideas for your next Linux hardware hack - just don't forget to share the fun with LG readers! I can be contacted via my home page at pramode.net.
I am an instructor working for IC Software in Kerala, India. I would have loved
becoming an organic chemist, but I do the second best thing possible, which is
play with Linux and teach programming!
Object Caml is an ML type of language. For the non-gurus: it's a functional language that can also be programmed in a non-functional and object-oriented way.
This language is really easy to learn. It's powerful and keeps impressing me with its speed. Programs written in this language are almost always stable by default. No segmentation faults, only occasional unending loops for the programmers that still hang on to program their own loops. It is really not needed to write most loops, since the libraries contain standard functions that are good enough in 99% of the cases. So try to use those functions: It really pays off in terms of stability of your programs, and, unless you have intimate knowledge of the inner works of this language, they tend to be better optimised.
The language can be obtained from the website caml.inria.fr. Here, they provide RPMs for the RedHat 7.2/8.0/9 and Mandrake 8.0 distributions. Also MS Windows binaries are available, but not all Unix library functions will work there, for some mysterious reason. The source tarball does compile flawlessly for me. It just has a somewhat unusual makefile layout:
# ./configure # make world; make opt; make install
The normal libraries include many usable data-structures like balanced trees, hash tables, and streams. Their version of header files (.mli files) contain all the basic documentation you need, and those are directly converted into HTML and published on the Web in their OCaml manual. This manual is not very usable to study this language, so I'll try to explain here some of the basic language constructions. This is just to give you an impression of the power of this language.
Now some real life examples. I wrote a program to help administrating a computer. It is a subset of a normal file finder, but is a command line tool and very fast. It helps locating large, not-recently-used files to be deleted from the system. It crawls through the directory tree and show the contents in different layouts.
Every module in OCaml has its own namespace. Specific definitions can be found by adding the module name, with the first character an upper-case character. You can also change the namespace of the current program to include a total module. Normally, only the standard module 'pervasives.mli' is included in the default namespace. The example program 'show.ml' starts with:
open Basics open Unix open Unix.LargeFile
This includes my own set of 'basics' functions and 2 standard libraries: 'Unix' and 'Unix.LargeFile'. A module normally consists of 2 files. The first file for exporting definitions 'module.mli' (like the C .h file), and the second one for actual code (the 'module.ml' file). The program uses the function 'string_sub' that provides a foolproof version of the 'String.sub' standard function (from the string.mli module). The basics.mli file contains the lines:
val string_sub: string -> int -> int -> string (** Get the sub string from a [string] from position [from] with [length]. This is the same function as String.sub, but it will never raise an exception. And a negative [from] value is counted from the right side of the string. *)
This gives the definition of this function and the description. There is an automatic documentation generator (ocamldoc) that reads .mli files and writes .html files as basic interface documentation. Normal comments start with (* but the documentation generator only writes comments that start with (** to the .html files. This document contains links to the documentation of the used modules. This documentation is really helpful to start programming ocaml. The .mli files are all included in the distribution, but the complete manual and a book can be downloaded from the Web site caml.inria.fr
The function is followed by its type. It wants 3 parameters and provides a string. Normally we need to write 'Basics.string_sub' to use this function. But after the 'open Basics' instruction just 'string_sub' is enough.
Now, back to the main program again. The first function is 'gettype'. It will try to return the type of a file. The file type is defined as the part of the filename following the last '.'. When there is no dot, the type is unknown and returned empty.
let gettype file = try let pos = String.rindex file '.' in String.sub file (pos+1) (String.length file-pos-1) with Not_found -> "" ;;
This function only uses standard functions. First, it catches the Not_found exception in the 'try' 'with Not_found -> ""' code. All other exceptions will be passed to the caller to be handled, and can possibly stop the main program. The local variable pos get is filled with the result of the function rindex. This function is also the reason to catch the exception; otherwise, the main program might stop on the first found file with no '.' in it. Local variables can be declared everywhere inside ocaml with 'let <variable> = <value> in <code>'. After the completion of the given code, the variable is out of scope and will be forgotten. The data will be passed to the garbage collector to be removed from memory. Function calls do normally use brackets. The function call to 'String.sub' gets 3 parameters the string 'file' the integer '(pos+1)' and the integer '(String.length file-pos-1)'. The last parameter calls the function 'String.length' with a single parameter 'file'. So, the functions are eager for their parameters; brackets are needed only when the parameters are filled with calculations.
Also '(+)' and '(-)' are functions of the pervasives module. It is very easy to define your own operators; just add brackets around their definition, and they are ready.
The next routine 'filesize' in the example code is far longer, but largely introduces sub-functions and 'if <bool-expr> then <expr> else <expr>' statements. This function creates a string from an int64 number for human readable file and directory sizes. The types of parameters are normally not given; they are determined by ocaml through their usage. When something is not clear, the compiler or interpreter will complain about it before executing the code.
let filesize s = let tostr f = if f>9.9 then string_of_int (int_of_float (f +. 0.5)) else let res = string_of_float (floor (f *. 10.0 +. 0.5) /. 10.0) in if String.length res=2 then res ^ "0" else res in let bytes = Int64.to_float s in if bytes > 512.0 then let kb = bytes /. 1024.0 in if kb > 512.0 then let mb = kb /. 1024.0 in if mb > 512.0 then let gb = mb /. 1024.0 in tostr gb ^ " Gb" else tostr mb ^ " Mb" else tostr kb ^ " kb" else Int64.to_string s ;;
The ocaml standard library has a set of conversion functions. These functions normally follow the form of 'int_of_float' and 'string_of_float'. Specific types like 'Int64' use shorthand notations like 'Int64.to_float'. String concatenations are done with the operation '(^)'. Normally, functions are defined for only one specific type, so there are new sets of arithmetic functions for floats like '(+.)', '(*.)' and '(/.)'. The 'tostr' sub-function has some extra calculation to change something like '5. Gb' into the nicer form of '5.0 Gb'.
The next function, 'converttime', converts a string into a float. OCaml uses floats for date for 2 reasons. The first is to prevent possible Year 2k problems, and can also be used for less than one-second time measurements. The function accepts English acronyms for month names. So let's introduce the list and the pair to create a translation of acronyms into numbers.
let month = [("jan", 0); ("feb", 1); ("mar", 2); ("apr", 3); ("may", 4); ("jun", 5); ("jul", 6); ("aug", 7); ("sep", 8); ("oct", 9); ("nov", 10); ("dec", 11)] ;;
This list is totally static, and can be used easily by the standard function List.assoc to convert a string into the corresponding number.
let converttime str = try begin match if str>"a" && str<"z" then ( int_of_string (string_sub str (String.rindex str ' '+1) 99), List.assoc (string_sub str 0 3) month, 1 ) else ( int_of_string (string_sub str 0 ( try String.index str '-' with Not_found -> 99 )), ( try let pos=String.index str '-'+1 in int_of_string (string_sub str pos ( try String.index_from str pos '-'-pos with err -> 99 ))-1 with err -> 0 ), ( try let pos=String.index str '-'+1 in int_of_string (string_sub str (String.index_from str pos '-'+1) 99) with err -> 1 ) ) with (yr,mn,md) -> (* print_string ("Last access before: "^ string_of_int (if yr<50 then yr+2000 else if yr<100 then yr+1900 else yr)^"-"^ string_of_int (mn+1)^"-"^ string_of_int md^"\n"); *) fst (mktime { tm_sec = 0; tm_min = 0; tm_hour = 0; tm_mday = md; tm_mon = mn; tm_year = if yr<50 then yr+100 else if yr<100 then yr else yr-1900; tm_wday = 0; tm_yday = 0; tm_isdst = false }) end with err -> print_string ("Cannot decipher this date string '" ^ str ^ "'\n"); max_float ;;
The new operation in this function is the 'match <expr> with <template> -> expr'. This is one of the most versatile instructions of ocaml. It can be used to examine the contents of variables and get the needed information out of it. This function creates the triplet (year, month, day-of-month) out of 2 different date notations. To debug this function the 'print_string' instruction is included but commented out to prevent clutter in the output of the program. Normally there is some logging mechanism to make the extra messages optional for the user. The 'print_string' shows the ISO notation of the given date; it creates a 4-digits year and gives a month number with January=1 instead of the internal Unix use of January=0.
This function also shows the use of 'try <expr> with err -> <expr>' that caches every possible exception and fills the variable 'err' with the details of the exception. This function can raise quite a lot of different exceptions, and frankly I am not very interested in the details. The routine just complains to the user about the given date string and gets over it. It returns the maximal possible float to include every filename.
The main standard function is the 'Unix.mktime' function. It wants to get a record filled with numbers about the current time. This function returns a pair with the needed float and a normalized record. With the pervasives function fst returns just the first parameter of the pair.
The ';' before the 'max_float' indicates that the expression results in a float, but the instructions before the ';' are calculated first. This is the first non-functional instruction inside the example code. OCaml is not strictly functional, but has the full power of other functional languages.
Now is the time for a real data structure that is dynamically build and can be used in a lot of different ways.
type entrytype = | Dir of entry list (* directory with a list of files *) | File of string (* a file inside a directory *) and entry = { mutable e_name: string; (* name of a file or directory *) e_type: entrytype; (* what type is this together with type related information *) e_atime: float; (* last access time *) e_size: int64; (* size of the file or size of all the matching files in the directory *) }
The 'and' statement is used to glue the two definitions together. They are created at the same time so that 'entrytype' can include 'entry' and vice-versa. 'entrytype' can consist one of 2 things: a directory with a list of entries or a file with its type. The directory entry has a mutable name. This is can be used later on to change a filename info the full path to that file.
As with ANSI C, the operators for Boolean algebra are '(&&)' and '(||)'.
let rec dirwrite el depth sortfn = List.iter ( fun e -> match e.e_type with | Dir lst -> if e.e_size <> Int64.of_int 0 then begin print_string ((String.make (depth*2) ' ') ^ "Directory " ^ e.e_name ^ " = (" ^ filesize e.e_size ^ ")\n"); dirwrite lst (depth+1) sortfn end | File string -> print_string ((String.make (depth*2) ' ') ^ e.e_name ^ " (" ^ filesize e.e_size ^ ")\n") ) (List.sort sortfn el) ;;
Here is the recursive ('rec') function 'dirwrite' that traverses a given tree 'el' and writes the result to the standard output. The parameter 'depth' indicates the amount of spaces to write a tree like structure of filenames. The function sorts all the lists with the given function 'sortfn'. The new language structure here is 'fun <parm-1> ... <parm-n> -> <expr>'. This construction creates a function without a name. The parameters of this function like construction can be used like a template to match pairs.
This function suppresses directories that are 0 bytes in size to reduce clutter.
(* List of global variables *) let min_size = ref (Int64.of_int 0) and (* minimum size of a file in bytes *) last_access = ref max_float and (* last access time in seconds since 1970 *) has_type = ref "" and (* type of file to show or empty to show all *) name_match = ref "" and (* regular expression to match the filename with; empty is show all *) name_regexp = ref (Str.regexp "") and (* pre-calculated regular expression *) no_symlinks = ref false (* don't follow symbolic links to directories *) ;;
This is a list of variables that can be changed due to the 'ref <expr>' construction. Normally definitions are just a label to their contents. These definitions are pointers to the memory and can be read by '!<variable>' and written by '<variable> := <expr>'. The parameters given to the program can make changes to the way the files are read.
let rec dirread path = let list = ref [] and size = ref (Int64.of_int 0) in try let dh = opendir path in while true do let file = readdir dh in if file<>".." && file<>"." && file<>"CVS" && String.sub file 0 1 <> "." then let s=stat (path^"/"^file) in if s.st_kind = S_DIR && (not !no_symlinks || (lstat (path^"/"^file)).st_kind <> S_LNK) then let dir = dirread (path^"/"^file) in list := { e_name = file; e_type = Dir (fst dir); e_atime = s.st_atime; e_size = snd dir } :: !list; size := Int64.add !size (snd dir) else if (!has_type = "" || gettype file = !has_type) && s.st_size > !min_size && s.st_atime < !last_access && (!name_match = "" || Str.string_match !name_regexp file 0) then begin list := { e_name = file; e_type = File (gettype file); e_atime = s.st_atime; e_size = s.st_size; } :: !list; size := Int64.add !size s.st_size end done; (!list, !size) with | End_of_file -> (!list, !size) | Unix_error (EACCES, err, parm) -> (!list, !size) ;;
The following functions are introduced in the function 'dirread':
let rec flat el path = List.fold_right ( fun e ls -> match e.e_type with | Dir lst -> flat lst (path ^ "/" ^ e.e_name) @ ls | File string -> e.e_name <- (path ^ "/" ^ e.e_name); e :: ls ) el [] ;;
This neat little routine 'flat' hits the tree 'el' flat on the ground. It takes every file from every branch and creates a single list of all the encountered files. This is done with one of the most versatile standard routines inside ocaml: the 'List.fold_right' routine. This routine introduces a state machine (scarab) that crawls over a list and operates on every encountered element. It produces a new structure (droppings) as a result -- in this case, a flattened list.
The construction '<record-field> <- <expr>' changes the contents of a mutable record field. Without mutable fields, you can mutate records only by creating a new one with lots of fields inherited from the old one. This is a shortcut for that.
let name_order a b = compare a.e_name b.e_name ;; let type_order a b = let typea = match a.e_type with Dir ls -> "dir" | File tp -> tp and typeb = match b.e_type with Dir ls -> "dir" | File tp -> tp in if compare typea typeb = 0 then compare a.e_name b.e_name else compare typea typeb ;; let atime_order a b = compare a.e_atime b.e_atime ;;
A set of sorting functions to use inside 'dirwrite'. The function 'compare' results in the widely used values of -1 for lower than, 0 for equal and +1 for higher than.
let dir = ref "." and sort = ref name_order and show = ref 0 in Arg.parse [ ("-t",Arg.Unit (fun () -> sort := type_order), "Sort by type and filename"); ("-l",Arg.Unit (fun () -> sort := atime_order), "Sort by last access time"); ("-n",Arg.Unit (fun () -> show := 1), "List filenames"); ("-b",Arg.Unit (fun () -> show := 2), "List both filenames and sizes"); ("-s",Arg.Unit (fun () -> no_symlinks := true), "Don't follow symbolic links"); ("--before",Arg.String (fun s -> last_access := converttime s), "Last access older than give date (format 'yyyy-mm-dd' or 'mmm yyyy')"); ("--size",Arg.Int (fun i -> min_size := Int64.mul (Int64.of_int i) (Int64.of_int (1024*1024)) ), "File size bigger than size in Mbytes"); ("--type",Arg.String (fun s -> has_type := s), "File is specific type"); ("--name",Arg.String (fun s -> name_match := s; name_regexp := Str.regexp (s ^ "$") ), "Filename matches regular expression") ] (fun d -> dir := d) "show [DIR]"; let res = dirread !dir in if !show=0 then begin dirwrite (fst res) 0 !sort; print_string ("Total size " ^ filesize (snd res) ^ "\n") end else List.iter (fun e -> print_endline (e.e_name ^ if !show=2 then " ("^filesize e.e_size^")" else "") ) (List.sort !sort (flat (fst res) !dir)) ;;
And here is the main routine. It calls the Arg.parse routine to parse the parameters given to the program. But this is too much un-GNU for me. I wrote my own version of it that follows the GNU coding standards a bit more than the default one (Gnuarg). The other version is a bit more complicated so I will include only the sources that use it.
The code can be obtained from here. Just unpack it somewhere with 'tar -xzf show.tar.gz' and move into the source directory with 'cd show/src'. There is also a Makefile that compiles to machine code and installs everything. But Makefiles are too rough for sour eyes to show in this article. The nitty-gritty details are there in the source. The general compile form is.
ocamlopt -o show unix.cmxa str.cmxa basics.cmx show.ml
The only non-standard libraries in use here are unix.cmxa and str.cmxa.
make su make install exit show --help show -s ~ --size 3 --before "apr 2003"
That concludes this example program.
Developer at a small technology firm in the Netherlands called V&S bv.
(www.v-s.nl)
We sell firewall, anti-virus and spam boxes based on the Linux OS.
Using more and more the OCaml language to write my applications.
Busy writing a lightweight http server with an internal scripting language
(camlserv.sourceforge.net,
source code here)
Interested in writing AI based computer games. Always trying writing
one, nothing ready yet.