Posts Tagged ‘command-line’
command-line, instructions, irc, learning, linux, software, software freedom, technology, ubuntu
In learning, linux on May 26, 2010 at 3:28 pm
Next task: make a bootable USB thumb/stick/drive/thing to rescue a busted machine (In this case, little old Brahms). dkg, as ever, knows what’s what and gave me great advice:
Bootable USB sticks are just like bootable hard drives for modern computers. Partition them with parted, use mkfs to create a filesystem on them, use grub-install to give them a bootloader, put a kernel and an initial ramfs on them, configure the bootloader to load them, and away you go.
He even offered me his filesystem rescue debirf image with the latest 686 kernel from debian unstable. Read the rest of this entry »
command-line, linux, ubuntu
In various on December 15, 2009 at 3:00 pm
Okay: so I can play DVDs now, but they’re choppy.
Meanwhile, Ubuntu’s System Monitor shows 2.9 GiB of available memory, though I ought to have 4. The bios shows 4. What gives?
command-line, learning, linux, software freedom, technology, ubuntu, whining
In linux on December 13, 2009 at 6:34 pm
Okay, lazyweb:
Why can’t I play DVDs and why, when I have installed in my computer two RAM modules of 2GiB each, does my computer have 2.9 GiB of the RAMS? Why?
On the DVD end, I have installed one million things and gotten from an error in Totem (“no uri handler implemented for dvd”) to Totem quietly crashing when I try to play a DVD. VLC spins the disk and then stops. /var/log/messages shows this after trying to run VLC:
Dec 13 13:10:02 luna kernel: [10853.901791] sr 1:0:0:0: [sr0] Add. Sense: Media region code is mismatched to logical unit region
Dec 13 13:10:02 luna kernel: [10853.906376] sr 1:0:0:0: [sr0] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Dec 13 13:10:02 luna kernel: [10853.906386] sr 1:0:0:0: [sr0] Sense Key : Illegal Request [current]
Dec 13 13:10:02 luna kernel: [10853.906397] Info fld=0x98d0
Dec 13 13:10:02 luna kernel: [10853.906402] sr 1:0:0:0: [sr0] Add. Sense: Media region code is mismatched to logical unit region
Read the rest of this entry »
command-line, linux, software freedom, technology
In learning, linux on May 19, 2009 at 4:12 pm
We’ve got two packages running, OpenX and Phorum, that seem to make a lot of database connections. Sometimes so many that the whole database runs out of connections. Phorum gets hammered by bots looking to exploit vulnerabilities in the code (and sometimes finds them). OpenX is just greedy. Or needy.
So I got smart. I limited the database users that those tools connect as to just a handful of connections. Which solved one problem: we stopped getting errors complaining that there were too many users connected to the database. But we started finding that the site would slow to an absolute crawl from time to time.
Talking it through with a friend, he pointed out that what I was probably doing was causing all the rejected database connections to queue up and wait. Kind of not the ideal solution, though I bet I could further fine tune things to prevent that as well. A better idea, since the Phorum forums are entirely archival at this point, was to restrict any and all post requests in the directories where Phorum is running. In htaccess you’d say something like:
<IfModule mod_rewrite.c>
Options +FollowSymlinks
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(POST) [NC]
RewriteRule ^(.*)$ – [F]
</IfModule>
So that’s where I’ve left things for the moment. We’ll see how we fair.
PS. How much do I love that you can make a firefox search bar out of anything?
command-line, linux, myself, software freedom, technology, ubuntu
In various on April 15, 2009 at 10:43 pm
Forget for a moment that I’m trying to play like I’m more than a tech. Or recall Joanne this morning in the rain in the garden after Lucia and I collected coconut shells to mark out an area to plant her onions in, “so funny, computers seem like the last thing I’d imagine you working on. You’re someone who should be working outside! I can’t imagine you not working with plants!”
If only she knew.
So I have a PDF. PDFedit will convert it to a pretty crappy XML document. A round of non-greedy vim searches:
:%s:<font .\{-}>::g
:%s:</font>::g
:%s: bbox=”.\{-}”::g
Followed by some attention to the numbers:
:%s:period:. :g
:%s:comma:,:g
:%s:zero:0:g
:%s:one:8:g
:%s:two:2:g
:%s:three:3:g
:%s:four:4:g
:%s:five:5:g
:%s:six:6:g
:%s:seven:7:g
:%s:eight:8:g
:%s:nine:9:g
(and a bonus to anyone who can tell me how I totally borked my data with the series above …) got me a really, really simple XML file of “lines” and “words”. Read the rest of this entry »
command-line, mapping, technology
In various on February 5, 2009 at 3:31 am
Observation: I’m talking about feeding the borg here in a pretty big way. But I don’t know that I have it in me to totally DIY this.
Right now, when we post a new article, it is up to the editor to go through and identify all the hyperlinks that belong in the article. Then the web producer takes the article and runs it through some HTML filters and actually anchors those links. Someone, usually, produces a list of keywords. It is all a little arbitrary, the keywords part.
You might already know this, but arbitrary makes me insane. Seriously. I need structure. And, we’re working on doing a better job of covering elections and so one thing I want is a scheme for tagging stories about a particular district. Also, there are a lot of geographic news aggregators out there. EveryBlock, Outside.in and I’d like to get our stories onto their maps more consistently. And, we have these district pages about our elected representatives, and they’re pretty thorough, but instinctively writers and editors link off to electeds’ own home pages instead of our internal pages about them, which isn’t the end of the world but doesn’t do a very good job of keeping people on our site.
So what I want, is to take a story that is still raw, and process it a few times, in an automated sort of a fashion, before it gets laid out. I’d like, for instance, to search for places and geocode them. Search for elected officials and candidates and link to our own pages about those people. And at least on the geocoding front someone said I could “just use pipes.” And so I’m trying to figure out how that would work. I’m pretty sure it wouldn’t. I’m pretty sure I’d need to create a database of locations myself before Yahoo pipes would even begin to try to put the Adam Clayton Powell State Office Building or the Board of Elections on a map.
But that is what I’m thinking about.
command-line, myself, software freedom, technology, whining
In various on February 4, 2009 at 12:12 am
One: kid brother is trying to password protect photos of our nephew. Our nephew. Kid brother did not bear spawn. But he’s struggling with htaccess and authentication. Among other things that make it kind of difficult (besides just starting from scratch being difficult) he’s stuck with FTP access. So here is the story, loosely.
Let’s say you have a directory. We’ll call it “hamish” just because that is a nice word. Yiddish, I think, for “cozy” or “familiar.” So you have a folder called /srv/apache2/example.com/hamish/ and you want to password protect it. You need a password file. You can make it in a text editor, call it .htpasswd and stick a line in it like:
waffle:N.KP5MOATfaew
If you know what a waffle is, or was, species-wise, you’ll be able to work with me. Otherwise, just figure someone’s user name is waffle and their password is not maple syrup. Unless it is. You could just use that line for now, or you can generate your own using the htpasswd command or using a random web gui. The latter method does present certain security issues, but it is convenient. The other file you’re going to need is the .htaccess file, and assuming your .htpasswd looks like mine and is called .htpasswd and lives in /srv/apache2/example.com/hamish/ then your .htaccess will look about like this:
AuthType Basic
AuthName “Not Just Anyone Can See This”
AuthUserFile /srv/apache2/example.com/hamish/.htpasswd
Require user waffle
Upload those two files and you’re good to go. Anyone who knows from waffle will be able to log in. It is true that you can do a lot, possibly even a shitton, with htaccess, but the easiest place to start is basic authentication.
Two: whining. Which goes about like so: last night, we were through debating the relative merits of naming an unnamed collaborative project Palante Tech (we decided against usurping Palante just yet) and we were almost through debating Flickr and Facebook and privacy and I said something about Gallery and how it frustrates me that I can’t get webdav to work. And dkg said something like “well, if you’re using F-Stop you can just export directly to Gallery. Why not do it that way?” I was just whining. I didn’t expect a solution. But sure enough, if you enable the remote module in Gallery 2 (and probably Galleries one and three as well) you can use f-spot to tag and name and order your photos and then export them to your gallery. I don’t really have many photos that aren’t of my wedding, so I haven’t uploaded any yet, but I’m definitely done storing my own photos on Flickr as of yesterday. Meantime, I’m uploading even more photos of Vermont fields.
Maybe I’ll eventually get rid of that dagged WARNING: block core.NavigationLinks doesn't exist. error, too.
If I whine about tags and tagging in WordPress will someone fix that for me, too? I want to choose from my existing tags. Grump.
command-line, linux, software freedom, teaching, technology, ubuntu
In various on December 5, 2008 at 1:19 am
Someone (Hsuan, to be precise, but that isn’t the point really) came to me recently with a corrupted SD card. Normally, I don’t do corrupted SD cards. I so do not want to be that guy that sits there and fixes what is broke. That doesn’t appeal to me at all. But when there is a command line involved, or a real mystery … I can find it hard to resist. You can see why my career as something other than a widget twiddler is not really taking off.
The answer, if you like to skip the narrative (hmph) is photorec. Which actually does amazing things for all manner of data recovery.
Here’s what I had before me:
[0 amanda@stillwell CANON_DC]$ find . -type f
find: Filesystem loop detected; `./DCIM/101CANON/101CANON' has the same device number and inode as a directory which is 1 level higher in the filesystem hierarchy.
So whatever, PhotoRec. Great. But people always want to know why. I do. So why? Why? David Henry, who I do not know, and yet kind of do know (the internet is sneaky that way) had a pretty good explanation, which I can’t really improve on:
I was long puzzled by filesystem stuff, until I realized much is done by elves!
Read the rest of this entry »
command-line, learning, linux, technology
In various on December 2, 2008 at 9:12 pm
Worth trying: sar -q for a rundown of queue lengths and load averages. Also pstree and I need to sort out what this:
I reniced the gzip process with a low priority level to reduce the load it’s putting on your server. I’d recommend running these scripts with a +19 niceness during periods low traffic to reduce the impact they’re having on your server.
means.
command-line, learning, sed, technology, whining
In various on September 4, 2008 at 6:12 pm
What if I had a shitton of ancient shtml files that all, somewhere near the top of the body, contained a line like “BEGIN MAIN CONTENT”?
What if I wanted to wipe everything above that and substitute some include script? I’d use sed, right? Read the rest of this entry »
command-line, databases, learning, linux, software freedom, teaching
In various on August 22, 2008 at 10:53 pm
Observation: OpenOffice’s native database format is a pain. It is impossible to work with, impossible it import data into. A pain.
People keep telling me that OOo will talk to MySQL, but I haven’t ever bothered to sort it out. Lately, though, some guy at NYPIRG has been working magic with campaign finance data and I wanted a piece of the action. I turns out that the state Board of Elections does publish filing data regularly. Comma delimited no less. Or, rather, comma delimited data stored in a MS Windows only proprietary compression format.
I know I’m famous for saying that PHPmyAdmin makes you weak, but sometimes some weakness is called for. Especially when you’ve got a monstrosity of data and you want to be able to look around at it in a halfway useful way.
I’ve looked at the instructions before a few times over and never felt quite inspired to actually follow them, but with inspiration in hand I went for it this week. It took all week, but that had a lot more to do with trying to sort out a seamless way to modify the date formats for import into MySQL than with the OOo Base / MySQL connection.
To start with, I followed these instructions for installing Connector/ODBC. Or rather, I ran sudo aptitude install unixodbc libmyodbc unixodbc-bin and took it from there.
Most of the process you only need to do once. Then comes the fussy part. With Connector/ODBC in place, adding an existing MySQL database as a data source available to OOo Base requires you to:
run sudo ODBCConfig
look under the System DSN tab for the add button
select the MySQL ODBC Driver you configured the first time around and say OK
manually add the name of your database (it might not be in the pulldown)
Only then, can you use OOo Base’s New -> Database wizard to
connect to an existing MySQL database;
connect using ODBC;
and then select the database and supply a username and password for it.
It is a little crazy making to plod through the ODBC Config screens, but it does work.
command-line, php, technology, whining
In various on July 28, 2008 at 12:53 pm
Okay, so I’m getting someplace. I’ve got my chompy stuffs into an array. I’ve got it all in PHP. But now I need to sort a multi-dimensional array by a value in the second array. I’ve got an array (events) that contains each event as an array. And I want to sort by the date of each event.
<?php
#$datafile = $_SERVER['DOCUMENT_ROOT']."[...]/featured_data.pl";
$datafile = "testdata.pl";
$upcoming = array();
if(is_readable($datafile)) {
$lines = file($datafile);
foreach($lines as $line) {
$explody = explode("|",$line);
array_push($upcoming, $explody);
}
print "<!--";
print_r ($upcoming);
print "-->";
$ordered_events = array_reverse($upcoming);
foreach ($ordered_events as $value) {
$str ="> <a href=\"/http://www.abcnorio.org/pcgi-bin/suite/calendar/calendar.cgi?request=detail &website=default&event_id=".$value[22]."\">";
$str .= $value[8];
$str .= "</a> ";
if($value[0] == $value[1]){
$str .= $value[2];
} elseif($value[1]-$value[0] == 365) {
$str .= " every " . $value[5] . " ";
} else {
$str .= $value[2] . " through " .$value[3];
}
$str .= " (".$value[10].")";
$str .= "<br><br>";
echo $str;
}
} else {
echo "Could not open ". $datafile ." for reading.";
}
?>
command-line, learning, linux, postfix, technology
In various on June 5, 2008 at 4:53 pm
The Frankenserver walks again …
I finally figured out that the frankenserver, which serves web service for example.com and examplefoundation.org doesn’t know that it is not the mail server for these domains. The whole rest of the world knows it, but not the frankenserver.
I’m trying to figure out the right way to tell ye olde frankenserver (or perhaps postfix) that mail must leave the house. To be more precise: if WordPress (or OpenX or Urchin) generates mail for amanda@example.com, that mail can be delivered to some other address as defined in /etc/aliases, but it can’t be delivered to amanda’s actual mailbox off on Dreamhost.
If /etc/aliases sayeth “amanda amanda@gmail.com” then mail generated on the server for amanda will be punted to gmail, no problem. But if amanda isn’t defined the mail evaporates into the ether (or something). I want postfix, locally, to know that it should treat mail for example.com like any other mail–look up its MX record and send it there.
I’m not sure how to convey that to Postfix, however.
command-line, learning
In various on June 5, 2008 at 3:13 am
I have this database all defined for postgres. It is complex beyond necessity. Waaay beyond necessity, and it is making me all mad. I’ve been commenting things out right and left just to get something to work somehow.
Because I’m so nice, I’m sharing with you, below, all 580 lines. You can see my mad commenting. What I really want to know is if there is an easier way? Why are these constraint references breaking down? I’m not used to constraints. Also, does postgres just not need commas?
Read the rest of this entry »
Brooklyn, command-line, NYC, radio
In various on June 5, 2008 at 1:07 am
Here is what happened: I worked too late for no good reason. I looked over at the clock as I was turning on the radio and thought to my self, “crap. It is 8:03. I have an entire hour before Fair Game is over. It really is only on when I want to listen to the news. Grumble grumble grumble.” I turned on the radio anyway and was pleasantly surprised to find On Point on the air instead. It isn’t my very favorite show ever, but I like it enough and when I’m home from work and ready to make some food or play scrabulous and listen to substantive news coverage, I definitely prefer Tom Ashbrook to Faith Salie. It looks from the WNYC schedule like a permanent change.
So whoever did this for me, I give to you a too tight hug and a giant kiss.
Also, I’m monkeying with this blasted postgres create statement that won’t translate properly to mysql, and to make things just that much trickier it has some kind of monkey business with the line breaks. The line breaks look just fine to the naked vim eye, but mysql was giving me errors with line numbers off. So for instance, this table definition (those are line numbers),
123 DROP TABLE IF EXISTS share_types;
124 CREATE TABLE share_types (
125 id serial not null primary key,
126 csa_id integer not null
127 CONSTRAINT share_type_csa_id_fk
128 REFERENCES csas(id)
129 ON UPDATE cascade
130 ON DELETE cascade,
131 share_type varchar(50) not null,
132 account_id integer not null
133 CONSTRAINT share_types_account_fk
134 REFERENCES accounts(id),
135 notes text,
136 updated_at timestamp not null default current_timestamp,
137 created_at timestamp DEFAULT 0,
138 position int,
139 CONSTRAINT share_type_csa_id_unq unique (csa_id, share_type)
140 )ENGINE=InnoDB;
Would generate an error like …
ERROR 1064 (42000) at line 124: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CONSTRAINT share_type_csa_id_fk
REFERENCES csas(id)
' at line 4
Why line 124 when the problem is at line 128? There were other signs, too. So I did this:
:%s/\n/^M/g
and now the line breaks are good. Whew!
command-line, food, learning, scheming, sustainability, teaching
In various on May 22, 2008 at 7:12 pm
I’ve got two things up my sleeves:
- it’s what’s for dinner. When I was little, sometimes we were not allowed to ask what was for dinner. I never understood. Now I understand. Sometimes when you’re cooking dinner you don’t want to say what you are making because that could open the door to suggestions you don’t want to hear. Sometimes. Sometimes if I’ve been getting a lot of sleep and practicing yoga and capoeira and reading fiction and generally not stressing myself out, I don’t have this problem. My mother says that if I’m going to be nice to my eventual children I probably have to be less stressed out. She knows a thing or two, my mother does. Meantime, I’m thinking of starting a recipe blog about just eating food. Arif has one, it is pretty good. I’m secretly thinking of setting up a wordpress multi-author blog for the Clinton Hill CSA to allow lots of members to maintain cooking blogs.
- Hide and Sed with Reggie’s Ex or maybe Ask me anything, I’m a regular expert. Either way, you submit a question about whether it is possible to transform some heap of data and I’ll suggest a few ways to do it with regular expressions or Calc functions. Doesn’t that sound like fun?
PS, I don’t know why my stupid tag cloud doesn’t work. I can’t figure it out. I’ve tried and tried. Grr.
PPS, Speaking of eventual children (and no, this isn’t a veiled message or anything) some time ago I took to keeping eventual children at bay through a careful program of Fertility Awareness. I gave up eventually–it is one thing to take your temperature every morning at the same exact time if you’re trying to figure out why you keep getting your period when you’re looking to get knocked up. It is another thing to take your temperature every morning at the same exact time for five years with no end in sight just to avoid more conventional prophylactics. I’m glad I know a lot more about women’s health. I’m glad I put the thermometer back in the medicine cabinet, too. All of which is a big lead up to saying that Princess Melissa seems to be blogging again and since she is actively trying to get herself in a family way, her post on Fertility Awareness is hysterical. I think so anyway.
command-line, learning, linux, ruby_on_rails, technology
In various on April 22, 2008 at 1:56 pm
This is about programming, which only a programmer would know. Ruby is a programming language, rails is … rails. It is a library of code written in Ruby that you can hook all together in a row and make do things, quickly. So they say. Like so many things, most Ruby on Rails sites for beginners assume we all begin at the same place. I’m beginning at a place where this particular tip sheet is useful.
Our project is this: the Clinton Hill CSA has a database that works. It does things people need for it to do. I want to move it from its old host to our Dreamhost account, but since it won’t just launch, I need to understand what I’m looking at a lot better.
Soooo… Read the rest of this entry »
command-line, learning, linux
In various on April 21, 2008 at 3:33 pm
Assuming you’re using Apache. I’m using Apache2, so I can’t even promise this will work on 1.x. Assuming you’ve got a BSD or Linux system. I keep coming back to this and my own archives are so scattered it isn’t a lot of help.
You need two things:
1) A virtual host definition. Somewhere in the directory where your virtual hosts are stored, make another one. /etc/httpd/conf.d and /etc/apache2/sites-available are likely candidates.
2) A domain name defined. Here is where things get sneaky. You can put definitions in /etc/hosts, which I like to do for local domains. On our Rackspace account, I hafta head into Rackspace’s domain name manager to add a subdomain. I don’t know where that gets written to. There are probably another dozen ways to define domain names. These are the two I’ve been using that I keep trying to look up.
command-line, software, technology
In various on April 16, 2008 at 5:03 pm
After yet another edit collision on the Frankenserver’s homemade CMS which does no type of file locking, I’m looking one more time at Bricolage. There is something I don’t get, something about why it needs its own instance of Apache, but it seems to.
I don’t know why I missed this the first five times around, but it actually isn’t supposed to be impossible to do that, to run two instances of Apache but call one of them Bricolage. I still don’t like when programs speak their very own language, but I try to be a flexible person.
command-line, subversion, teaching, technology
In various on April 3, 2008 at 7:51 pm
Today, I was sneaking in to change the name of the Hardrive on an intern machine from “Macintosh HD” to “Dunk” since despite my best efforts to change things around here I still seem to be responsible for communicating with our mostly inaccessible freelance sys admin dude and I get tired of calling machines by an enumeration of past users (The machine that was most recently Mia’s but before that was Josh’s is about to get a new user, since Mia’s moved on to bigger and better things, but actually was it Oate’s workstation for a minute there?) so I’m naming machines after famous circus elephants. Maybe if I can persuade our MIFSAD to at least contemplate adding RAM to any of them I’ll move to fast cats gone wild (Montcore, Jorge, Tatiana), but … a colleague said something to me about catching me touching a Mac. I’ll grant that I’ve been pretty severe about insisting that I don’t do desktop support, but it isn’t because I’m a hater, it is because I can’t be our in house desktop support person and still do my job and I’d really rather do my job. No big thing.
I do have a Mac question, though, because my job will be easier when at least our graphic designer gets the hang of subversion. I tried to find for her a starting point like some instructions on the intertubes, but found instructions that called for Fink (noooooooooo) or involved building things from source (nooooooo). So I asked around instead. Here’s what I know so far:
- BBEdit supports SVN but only for one by one file editing — you can check out and edit one file and then check it back in, but you can’t update your local working copy. So it is a nice hook for once in a very rare while use, but it isn’t really a Subversion client.
- RapidSVN is a GUI SVN client, but it has
dependencies that include subversion itself which, of course, is available for MacOSX in eight ways.
- Over at Apperceptive, staff, techie and otherwise, use a command line svn OSX’s terminal app. They installed the collabnet package, though OSX Leopard ships with it already in place.
- Deanna uses Eclipse, which works just fine for her purposes, her purposes being mostly interfacing with repositories maintained elsewhere, which is pretty much what I’m after.
- Jonah suggests SmartSvn for people who need GUIs. It is java based and does have a gratis version but is not Free
as in software. Software freedom isn’t a requirement here (it is s’posed to run on OS X for crike’s sake) so I’ll take a look at that one.
- Matt has his students using the SCPlugin from Tigris. They’re all running Leopard, which has a command line client already installed. SCPlugin works from the Finder, as a contextual menu — it shows up under
More...-->Subversion. Once working copy is checked out, the user can to right-click on files to commit or revert. Browsing logs and rolling back aren’t really there (or aren’t all that fancy). Also via Matt, a nudge towards the collabnet Mac package for commandline SVN.
- Phillip uses SVN on a
Mac both via the terminal, and integrated with BBEdit. MacFuse works great for connecting to remote directories that are already under version control. Pre-leopard users will need the Mac package to run Subversion clients locally.
Here is what we actually did:
- Install the CollabNet binary
- Create a .bashrc that adds
/usr/local/bin to her path.
- Make a directory in Sites to contain our working directory.
svn checkout http://svn.example.com/svn/repository
command-line, postfix, security, technology
In various on February 10, 2008 at 6:30 am
I’m using iptables to block incoming requests from the machine out there who’s got my SMTP server on auto dial. sudo /sbin/iptables -A OUTPUT -d xx.xxx.xxx.xxx -j DROP. I know it isn’t the Right Way, but I want my logs back and I’m hoping he’ll eventually notice that he really isn’t getting through and stop trying. Could be wishful thinking, which makes it that much more ominous that I can’t get sudo /sbin/iptables -D INPUT 1 to work (I screwed up, see, and did INPUT not OUTPUT the first time ’round.)
PS. can someone please tell me what is so great about Emacs that I should be scorned for using Vim?
command-line, learning, linux, postfix, security, technology
In various on February 7, 2008 at 10:45 pm
Still kind of wondering if I could/should just close off smtp connections in an xinetd config, but tail /var/log/maillog seems to be saying that my little friend at 81.189.52.37 is having less success connecting of late:
Feb 7 17:35:59 s47822 postfix/smtpd[6537]: disconnect from unknown[81.189.52.37]
Feb 7 17:35:59 s47822 postfix/smtpd[6544]: disconnect from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6523]: lost connection after CONNECT from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6523]: disconnect from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6538]: lost connection after EHLO from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6538]: disconnect from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6522]: lost connection after CONNECT from unknown[81.189.52.37]
Feb 7 17:36:00 s42782 postfix/smtpd[6522]: disconnect from unknown[81.189.52.37]
Feb 7 17:36:01 s42782 postfix/smtpd[6506]: lost connection after EHLO from unknown[81.189.52.37]
Feb 7 17:36:01 s47s42782822 postfix/smtpd[6506]: disconnect from unknown[81.189.52.37]
Or am I missing something? This is an interesting list of main.cf settings to investigate. And this is a rundown of what they actually mean. Does adding reject_invalid_hostname or reject_non_fqdn_hostname do anything for me?
command-line, learning, linux, teaching, technology
In various on February 5, 2008 at 11:28 pm
From the kid brother files …
Forgive me if you already know this command but if you don’t it is really handy. script will allow you to record every keystroke in terminal until you type “exit” (I think it object if you try to enter vi or pico or stuff like that). This is handy if you are trying to solve some problem or download some packages in the terminal and want a record of what you have done. (just type $script filename.lst and it will record your activities – I’m sure that if you don’t already know about the command you are smart enough to figure out the rest)
Right off the bat, some observations: are you really using vi and not vim? Did you know that you can usually look up the manual page of any command line linux goodness with $ man {command}?
And, if you’re using bash as your shell, your command history is stored in .bash_history. Try typing history at the command line.
My problem is this: in the shell you can hit the up arrow to scroll through your last dozen commands (sis note: that is the history I speak of above.) but script records all these up arrows and makes lines on the final document that are hundreds of chars long. So, is there a command that escapes this recording and just records the final command you settled on rather than the dozen you scrolled through to get to it?
It doesn’t look like script will let you do much besides log your keystrokes. What kind of packages are you downloading? I’ve been using aptitude over apt precisely because it does a much better job of showing you what you installed. Not that I remember off the top of my head how I did that or anything. But I did once. See such a list.
Are you downloading with wget? Try history | grep wget when you’re done downloading things. That ought to give you a good list of what you wgot.
Other fun facts? ctrl-r will search your history for you, so if you say ctrl-r and then start typing a command, it will offer you recent command history that matches what you are typing. Play with it. You can customize the number of lines that are stored in your bash history. Here’s that section of my .bashrc file:
# don't put duplicate lines in the history
export HISTCONTROL=ignoredups
export HISTCONTROL=erasedups
export HISTCONTROL=ignoreboth
# store more lines in history
export HISTFILESIZE=5000
# and don't store dumb stuff
export HISTIGNORE=pwd:exit:cd:ls:history
I think I drifted before I ever got the part about not logging duplicate commands to work as expected, but you get the idea.
command-line, learning, linux, technology
In various on January 30, 2008 at 10:27 pm
Another good reason to use ssh keys? My key is still on the server of a group I haven’t worked with in three years. I have no idea where I’d start looking for a password, but when the called me in a panic because their site had been “oWneD bY soM3 dw33b” I had no trouble getting onto their server. That might not be what you want, for me to be leaving keys under mats on servers the world over, but it is surely convenient. Passwords are tiresome.
Since I always end up having to look this up, here is a fun secret: for key based authentication to work, your .ssh directory usually needs to be owner only access:
chmod 700 .ssh
If you don’t know, now you know.
command-line, technology, whining
In various on January 29, 2008 at 1:29 pm
Was the mustache game fun? (tell me I didn’t miss anything).
Will they who come after recognize that these little text files named “meep” and “feh” came from me? Is this my footprint?
Why does grep think that text files aren’t?
25 Binary file ./suggests/insert.php matches
26 Binary file ./suggests/insertcat.php matches
Is it grep what thinks that? Similarly, is there a batch operation that will let me fix the file formats of thousands of files with ^M where they mean to have line breaks? Read the rest of this entry »
command-line, technology
In various on January 16, 2008 at 8:15 pm
This is just plain weird, from my error logs for a site that about five people ought to be accessing:
[Wed Jan 16 14:48:39 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:39 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:40 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:40 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:41 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:41 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:42 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:42 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:43 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
[Wed Jan 16 14:48:43 2008] [info] (32)Broken pipe: core_output_filter: writing data to the network
Apache has a helpfile on this particular error, but I wouldn’t call it the most illuminating manuscript out there:
Invalid argument: core_output_filter: writing data to the network
Apache uses the sendfile syscall on platforms where it is available in order to speed sending of responses. Unfortunately, on some systems, Apache will detect the presence of sendfile at compile-time, even when it does not work properly. This happens most frequently when using network or other non-standard file-system.
Symptoms of this problem include the above message in the error log and zero-length responses to non-zero-sized files. The problem generally occurs only for static files, since dynamic content usually does not make use of sendfile.
To fix this problem, simply use the EnableSendfile directive to disable sendfile for all or part of your server. Also see the EnableMMAP, which can help with similar problems.
I’m using ext3, which I don’t think qualifies as a “network or other non-standard file-system.”
So what gives? I don’t know where to start. Read the rest of this entry »
command-line, technology
In various on December 26, 2007 at 7:17 pm
Another thing I repeat:
/^.\{5,100}$
Find short lines, but not blank ones. Most of those lines are headlines, so
:s/p>/h3>/g
Replace all occurrences of “p>” (like <p> and </p&g;t) with “h3>”.
Two problems: one, I can’t repeat the search (I can, but it takes two keystrokes, which is one too many for me) because the search term has been replaced by the search and replace term. two, I can repeat the search and replace action (okay, ditto: two key strokes, : and an up arrow) because it is an ex search.
There has to be a way to search for lines that match the first search term and execute the search/replace on lines that I agree are headlines. I just don’t know what it is. This is close, and a reasonable enough substitute, but it isn’t purefect.
:%s/<p>\(.\{5,100}\)<\/p>/<h3>\1<\/h3>/
command-line, linux
In various on December 21, 2007 at 12:39 am
One rule among many: whether or not it is more time-efficient, it is always more rewarding to write a script than it is to repeat just about anything.
If only I could script cleaning the toilet (oh wait, I never actually …)
Seriously, though. I want, specifically, to write a vim script, one that ought to be wildly simple. I want a script that kind of does this:
:%s/\([a-z]*\)[ ]*\n[ ]*\([a-z]*\)/\1 \2/g
:%s/[“”]/"/g
:%s/’/'/g
:%s/\n\n/< \/p>&<p>/g
:%s:http\://[-A-Za-z0-9.%?&$_\/=]* :<a href="&">:g
:%s/ ">/">/g</a></p>
Because that is what I keep doing. Maybe if I write that script, I’ll show you how.
Updated
As it turns out, this works just fine:
:%s/[“”]/"/g
:%s/’/'/g
:%s/…/.../g
:%s/–/--/g
:%s/\n[ ]*\n/< \/p>&<p>/g
:%s:http\://[-A-Za-z0-9.%?&$_\/=]* :<a href="&">:g
:%s/ ">/">/g
# :%s/\([a-z]\)[ ]*\n[ ]*\([a-z]\)/\1 \2/g</a></p>
That last line is commented out because it isn’t part of the regular routine, but I do need to have it handy.
command-line, linux, technology, ubuntu
In various on December 15, 2007 at 4:18 am
Kaiser’s rehab hospital goes by the acronym KFRC. If you grew up in San Francisco you know why that cracks me up. The rehab hospital has a farmers market out front with amazing tamales and enchiladas. This is something I miss about the Bay Area, in general, tamales at the farmers’ market. On that note, the Union City Raizes do Brasil batizado had a potluck afterwards with papusas, which are something else that just are not as easy to come by in New York City. He’ll be fine, that is becoming more and more clear.
If you, dear reader, ever find yourself or a loved one in a hospital, and you find yourself feeling disempowered or overwhelmed (or if you find you are feeling “just fine” which is actually a clear sign that you are in over your head) I have got loads of advice for you, because I am an expert in patient advocacy. I might know eff all about medicine, but I know from insisting that someone be cared for adequately. I have loads of suggestions about articulating your own needs (“Well, I could use a massage and here are three places that I know for a fact sell gift certificates”) and about the importance of getting a calendar set up to regulate visitor so that you don’t wind up mobbed one day and all sad and lonely the next.
Meanwhile, I am really taking a shine to sed -i -e 's/foo/bar/g' ./* except that I need it to recurse through directories not balk at the first directory it hits and then sit there. Since I also don’t want to search through .svn files, and since for bonus fun a lot of files that make up our site are sans file extension, I’m reading this and contemplating something with -exec.
I was kind of hoping that I could use a little bit of find ./ -not \( -name "*.svn-base"" -o -name "foo" \) '{}' \; -print but as it turns out what I really need to do is more like find ./ -wholename '*/.svn/*' -prune -o -exec grep "somestring" {} \;.
UPDATE (5 Mar 2008)
Here is what works: find ./ -wholename '*/.svn/*' -prune -o -type f -exec sed -i -e 's/free software/free free software/g' {} \;
Another UPDATE (3 Apr 2008)
Another search that works similarly (but without sed) grep -ir --exclude='*.svn*' "software freedom" /srv/apache2/
command-line, technology, ubuntu
In linux on December 12, 2007 at 5:13 am
Hypothetically, let us say that you just paid cash for a wireless router from a guy who goes by the name “Sir Yoda.”
Let us say then, that you were looking forward to using this router. Google knew the default password. Off to a good start. Anything plugged into the Barricade seems to be online. On a roll. The point of doing business with a guy named Yoda was to be able to take Brahms downstairs to where the table is big and you can eat ice cream without fussing with coasters. You want wireless.
What little bits I already worked out back in May got me as far as some errors to search on (errors like There is already a pid file /var/run/dhclient.ath0.pid with pid 14919232), which got me to:
sudo /etc/init.d/networking restart
Which worked. I’m connected, though sans encryption. Next challenge? Read the rest of this entry »
command-line, learning, linux
In various on November 26, 2007 at 4:10 am
I haven’t posted about wrangling with computers in a while because I haven’t really been wrangling with computers lately and people seem to take offense when you go and post about how infuriating they can be (at least as infuriating as software) so I’ve been mostly keeping my mouth shut.
Now, though, subversion is trying to kill me and causing me a headache and I find that when I post about these headaches here, one of two things happens. Either I figure out the problem in the course of posting and maybe post it anyway, maybe don’t. Or someone else reads my whining and offers me some fruitful suggestions. Or nothing happens, which would be three things.
I did something screwy and then tried to undo it and that didn’t work so I tried to redo that and undo something else and I didn’t really keep an especially careful accounting and now I’m basically screwed. That is kind of an overstatement but at the present moment my repository is all busted up and I don’t know how to fix it.
First off (not first off, but I think this is the small thing so I’m starting here) I’ve got a directory (/html/games/incs) that seems not to have a .svn/tmp directory. I don’t see one, and when I do …
[somebody@someplace incs]$ svn cleanup
svn: Can’t open directory ‘.svn/tmp’: No such file or directory
[somebody@someplace incs]$ mkdir .svn/tmp
[somebody@someplace incs]$ svn cleanup
I was going to say that I know I have permission to create .svn/tmp but then I went to check and lo and behold, not only can I create the dang directory but that problem looks solved. Sweet. Okay, onto the next one.
PS, In case it is eating at you, I know that my whole tag situation is all busted up. I don’t have any plans to fix it because I just don’t. I think about it. I know I ought to, but you can assume that I’ll be mailing my mother’s birthday present first and since that is sitting on my desk more than a week post-due … you do the math.
So, um, huh?
command-line, storytelling, whining
In various on October 25, 2007 at 1:11 am
Some of you know about my ongoing debate with dkg about documentation and public lists. Or publicly archived lists. We go round in circles about whether all lists should be publicly archived by default. In free software communities, public list archives are an important source of information, context and technical support. In political organizing communities, lists can be much more complex, places where people are exploring political and theoretical ideas that they might be embarrassed by later.
If I say something foolish and naive on a technology support list, and then years later a colleague finds it, people will understand. They’ll believe that I get it now, that my grasp on Apache configuration syntax has grown. If I say something foolish and naive on a political list, though, that can haunt me for years. Even if it doesn’t hurt me in some objectively measurable way, it can sting. Even if you aren’t worried about embarrassing yourself one day, it is a little creepy to realize that anyone with a web browser can assemble a short history of my life and career by following posts I’ve made to various mailing lists over the years. I don’t like it.
On top of that, I firmly believe that a bit of privacy facilitates frank and honest conversations. We can learn together as friends if we aren’t broadcasting our every exchange. I’m certainly more comfortable working that way. I treat my posts to large lists very differently than I treat small ones and on the whole I am much, much less likely to weigh in on a conversation that is widely archived because I feel more pressure to fine tune my thoughts and carefully edit my phrasing. I like to look good. I know not everyone does, or maybe I know that not everyone judges “looking good” on the same terms, but I take pride in my prose.
So we go back and forth, dkg and I. Read the rest of this entry »
command-line, linux, truth_in_advertising, whining
In various on October 24, 2007 at 6:01 pm
I manage some servers with a small group of guys here in En Why See. We just built a new server and so far, the version of Vim we are running doesn’t support color coding. I found some How To that actually makes the whole process sound redunkulously complicated when I’m about 100% sure there is an easier way. A package, I’m guessing.
I don’t really want a teachable moment here. I want someone to say “I think that you’re looking for vim-full. The vim package doesn’t include any of the color syntax files.” That is all. If you know, maybe you could fill me in. If you don’t know, no big thing. I don’t know either. Just thought I’d ask.
Read the rest of this entry »
command-line, learning, linux, ubuntu
In various on October 17, 2007 at 8:57 pm
Hypothetically, lets say your (GNU/linux) computer is choking. Sending errors about disk space, perhaps. Maybe a message from the mail server about /var/mail being full up, maybe a little pop up on your desktop that says “warning. 100% full”
Since Stillwell is the second machine in so many weeks to fill up on me, I’m going to rehash what I learned from Ignatz’s overload finally. And I won’t get surly, promise.
First question: is it really full?
[0 amanda@stillwell ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 2.9G 2.7G 0 100% /
varrun 189M 128K 189M 1% /var/run
varlock 189M 0 189M 0% /var/lock
procbususb 189M 108K 189M 1% /proc/bus/usb
udev 189M 108K 189M 1% /dev
devshm 189M 0 189M 0% /dev/shm
lrm 189M 33M 156M 18% /lib/modules/2.6.20-16-generic/volatile
/dev/sda1 99M 5.6M 89M 6% /media/sda1
/dev/sda5 30G 8.3G 20G 30% /home
/dev/sda6 30G 1.2G 27G 5% /mnt/vartemp
That would be a yes. If you want to talk about my freakadoo setup with /mnt/vartemp, you can, but in my book that is a story for another day.
Assuming I haven’t been paying attention to my computer, here is what I know from this report on my file system disk space usage: /home is on its own partition. So that isn’t the problem.
Also, I happen to know that /mnt/vartemp is where /var and /tmp live, though I had to place them there and then use symlinks to overcome a quirk of Ubuntu that creates trouble if /var isn’t on the root partition (grammar check anyone? better link?)
So I’ve confirmed the sensation of fullness. Now what? Maybe I want to see just what is filling things up.
[0 amanda@stillwell ~]$ sudo du -kx | sort -n
Ahoy, but that shows files in my home directory. Riiiight.
[0 amanda@stillwell ~]$ cd /
[0 amanda@stillwell /]$ sudo du -kx | sort -n
69188 ./lib/linux-restricted-modules
93636 ./usr/share/icons
103192 ./usr/lib/jvm/java-6-sun-1.6.0.02/jre/lib
104748 ./usr/lib/jvm/java-6-sun-1.6.0.02/jre
104784 ./usr/lib/jvm/java-6-sun-1.6.0.02
104792 ./usr/lib/jvm
114764 ./usr/share/fonts/truetype
127596 ./usr/src
131012 ./usr/share/gnome/help
131432 ./usr/share/gnome
134208 ./lib/modules
139804 ./usr/share/fonts
145572 ./usr/share/doc
153164 ./usr/lib/openoffice/program
190152 ./usr/bin
220580 ./usr/lib/openoffice
225204 ./lib
1019328 ./usr/lib
1066040 ./usr/share
2438632 ./usr
2720781 .
Nothing special there. I suspect that I just set the partition too small (‘specially since I was installing Amarok when it choked.) Next question? Was I smart enough to use LVM when I built this machine?
command-line, learning, linux, ubuntu, various, whining
In various on September 19, 2007 at 4:45 pm
I realized that I do have a smallish file that I’ve been tinkering with that would do for these purposes: We’ll call it SVG Sample.
I’m still annoyed that there isn’t any documentation of how Dia handles SVGs, but I’ll get over that. Or I won’t, but either way it is my problem, this inability to let go of small, small things.
Dia’s diagram tree shows exactly one thing (object? Is it an object?), “Standard Image” — it displays the various nodes all assembled together as a single image. It doesn’t recognize this as a group of images.
Inkscape recognizes the individual objects (nodes? are they nodes?) but fails to recognize the edges as connectors.
Kivio asks me to “select a filter” from the options “Kivio Document” and “XML Document.” The first (not surprisingly) fails with an error along the lines of “this isn’t a Kivio document” which is cool. We’re all on the same page so far. The latter filter then wants me to import an XSLT configuration but doesn’t show me any I can import. Not exactly a brick wall, but it isn’t getting me anywhere, either.
command-line, linux, technology, ubuntu, various, whining
In various on September 18, 2007 at 6:31 pm
I do these things to make myself crazy. I use command line tools and free software and I call it cool. From time to time, however, I hit against these walls where there is something I cannot do and cannot find my way around doing and I spent an eternity trying to do them or find documentation on them and still …
This is bordering on one of those rants where I’m all “I will punish you by not using your tools anymore if you don’t help me, dammit.” except that there is no one to punish and no one to yell at. Still, I may file this one under “Free Software Doesn’t Work” for the next time someone tells me that there is nothing (nothing!) that Windows can do that Linux doesn’t do better.
And then I wind up being scolded by some free software zealot as though I’m saying I can’t be bothered to figure out why my OOo Spellchecker uses british spelling. I want to edit a flowchart. I have it in graphml format and in svg format and both Inkscape and Dia will open the svg at least, but I can’t find anything like reliable information about how to get DIa to do more than show a single snapshot of the whole svg. I can’t muck with the individual nodes, which sort of defeats the point.
command-line, learning, various, whining
In various on September 14, 2007 at 9:33 pm
Inkscape.
Create an SVG flowchart in Graph::Easy, save the .svg and open it in Inkscape. Ungroup.
That is all. I was trying to do this in Dia, which was making me crazy because it wouldn’t let me ungroup the vector graphics that made up my flowchart.
command-line, learning, linux, ubuntu, various
In various on September 14, 2007 at 4:26 pm
Tinkering with creating a bootable USB disk with which to install Feisty on Brahms. Tried on round of effort and got to a boot error.
One how-to at my disposal suggests repartitioning the stick with cfdisk /dev/sdb
That got me to this fine error FATAL ERROR: Bad primary partition 0: Partition begins after end-of-disk Press any key to exit cfdisk, which I’m going to call helpful. Something didn’t work as planned.
Meanwhile, Oates can’t chop up a PDF in Acrobat Reader, but I can, because I’ve got pdftk. Which he could get, but I digress.
I got started, but realized I wasn’t naming things the way I ought and (of course) got to wondering why I don’t know how to use regular expressions in mv. If you want to take everything named *.pdf and rename it ABBR_{originalname}.pdf … how often do you want to do that? At least once every six months, right? So it is worth spending twenty minutes looking for a regular expression syntax that will save you a half a second. Right? (Please agree with me. I don’t know how I’ll handle it if you don’t …)
[0 amanda@stillwell ~]$ for f in *.pdf; do mv "$f" "ny_${f%}";done
It is a good starting point for doing more interesting things. Fortunately, I’ve done what I needed to do and I might never come back to this wee bit of shell scripting.
command-line, linux, various, whining
In various on September 12, 2007 at 6:58 pm
I’m trying to get Trac running, halfheartedly following some instructions for installing Trac on Dreamhost, but they don’t make any sense, so I tried some script thing that seemed to work, generally, but leaves me a foot short of the finish line. My installation doesn’t quite work, and I’m not sure where to go from here. Phooey.
I want to be able to manage (or at least view) a few different repositories and I’m hoping that when I get this all sorted out I’ll be able to do that. It would be nice if I could do that from within one Trac site, though I’m having doubts galore.
command-line, linux, security, ubuntu, various, whining
In various on September 12, 2007 at 6:32 pm
I’ve managed to avoid encrypting my email thus far because encryption isn’t secure, it is silly. Or I can make it sound silly if I talk about it enough. Plus I don’t do anything interesting. Now, though, I want help from this developer and he wants to communicate entirely in GnuPG form. He gets to decide that — I’m the one asking him for help.
Since I’m stuck doing this, I’ve gone ahead and posted my public key.
If you start using it, don’t expect miracles, since I check my email via IMAP from all kinds of places an my decoder ring doesn’t travel with me.
I’m secretly kind of confused, too. I can’t really explain what confuses me, which is never a good sign. I know that dkg wants a bit more from me, before he’s willing to trust that email really did come from yours truly. I am supposed to sign messages, but then you all’d have to get my public key (same key? different?) from a key server, or from me directly, to confirm my signature. I sort of understand some of this, you see, but there are layers of enough and I am not really sure how many I’ve peeled back.
command-line, learning, subversion, technology, ubuntu, various
In various on August 7, 2007 at 6:15 pm
Having thoroughly fouled up a repository in the process of trying to upgrade WordPress, I’m returning to the original project: nest my SVN repositories. I suspect that it is going to be easier than I originally thought. Starting with a broken repository, it goes like this:
Confirm that shit be haywire with …
[0 amanda@stillwell blogs] $ svn status
Move your wordpress files out of the way for safe keeping:
[0 amanda@stillwell blogs]$ mv wordpress /home/amanda/Desktop/wordpress
Try an svn cleanup without the stray directory:
[0 amanda@stillwell blogs]$ svn cleanup .
Now that things are unlocked (still not unlocked? You need the book.), get the original out of SVN (it is already gone from the filesystem, since you moved it)
[0 amanda@stillwell blogs]$ svn delete wordpress
[0 amanda@stillwell blogs]$ svn commit
Now, you’ve got a halfway clean slate and you can…
svn co http://svn.automattic.com/wordpress/branches/2.2/ wordpress
Copy your themes, plugins, wp-config.php and .htaccess over from your back up (or get new ones, if the new version requires them — don’t forget to think for yourself here …) and then get them talking to your original repository. You might even want to read the official instructions for more info on what you’ll want to keep.
We aren’t done here and I’m not convinced that this is going to work. Sigh. What I ended up with is three directories in “blogs” — plugins, themes and configs — that are part of the repository that I manage. So those are in my general repository’s subversion tree (is that a sentence?). I have copy them over to where they need to be in wordpress/wp-content/ when I change them. So there is a manual step, but we’re a lot closer than we were before. I am anyhow. Can’t speak for you.
You get the idea. If you don’t, you should do some reading.
Moving right along, both phpMyAdmin (which makes you weak) and Spam Karma 2 are inexplicably committed to opening some (but not all) php files as external documents. Blank ones. I blame my Apache config, but it still makes me tired.
PS. why, oh why does neither of these lines:
export HISTCONTROL=ignoredups
export HISTCONTROL=ignoreboth
effectively prevent my bash command history from logging a zillion duplicate command?
command-line, learning, linux, ubuntu, various
In various on August 3, 2007 at 8:50 pm
I use grep a lot to backtrace through the layers of includes that make up the site I manage, and since I’ve now got that site halfway into version control, I’m finding that I’m a wee bit frustrated because grep -r recurses through every directory, including the .svn folders. So I get a lot of extra responses. I want to tell grep not to look anywhere down the .svn path.
Or, perhaps, is there another tool I ought to try?
command-line, learning, linux, ubuntu, various
In various on July 30, 2007 at 5:53 pm
I know, I know. I promised about the puns. I’m not even frustrated.
Project du jour: version control with dreamhost one-click installs. When you update an installation using the Dreamhost control panel, Dreamhost moves your old files from directory to directory.old, and installs the update in directory. If you want these new files in subversion, you have to scoot all the existing .svn directories back into directory.I’ve never (really) written a shell script from scratch before and this seemed like a good place to start.
With the help of an indispensable overview of advanced bash scripting published by the Linux Documentation Project, I got this far:
#!/bin/bash
path=/home/username/staging/blogs/
oldblog=someblog.old
newblog=${oldblog/.old/}
directory=$path$oldblog
for file in $(find $directory -type d -name '.svn')
do
echo mv $file ${file/$oldblog/$newblog}
done
exit 0
I find all the .svn directories and print out a close approximation of the command that I want. A close approximation. What I still need to do is strip off the last .svn so that the result will look more like:
mv /home/username/staging/blogs/someblog.old/.svn /home/username/staging/blogs/someblog.old/
Once I’ve got that down I can actually execute the command itself instead of just writing it out. I think I’m missing something syntax-wise (I know it is in the manual someplace …) and while I puzzle over it, I need to move on to other things.
Still, if you’ve got feedback or a better way (more elegant? more right?) I’m all ears.
For bonus fun, my bash environment on Dreamhost insists on writing long commands to a single line. Why? I can’t read that shit.
command-line, learning, technology, various
In various on July 23, 2007 at 3:25 pm
I need to learn to resist puniness in my post titles, but I haven’t yet learned. I’m learning a wee bit about DNS, from dkg, because I’ve got some domains around that our local DNS server isn’t resolving properly. Or wasn’t. Now they are, but later, when they aren’t again, I’ll want to come back to this.
Here’s what dkg had to say: If the DNS servers offered by the DHCP server on your LAN are giving stale info, it’s probably worth tracking that down and reporting it.
Here are some handy ways to do it:
first, document that the suspect DNS server is giving the wrong IP address:
SUSPECT=x.y.z.w
KNOWNGOOD=4.2.2.1
QUERY=foo.example.net
dig @$SUSPECT $QUERY
dig @$KNOWNGOOD $QUERY
Or, rather:
dig 192.168.1.3 gothamgazette.net
dig 4.2.2.1 gothamgazette.net
then, you might also want to ask the authoritative nameserver for the relevant domain:
dig ns @$KNOWNGOOD $QUERY
dig ns 4.2.2.1 gothamgazette.net
if $QUERY is third-level domain (e.g. foo.example.net, instead of example.net), it may not give you an answer, but instead might give you an AUTHORITY section instead. You might then want to re-run the nameserver query against the second-level domain (e.g. example.net).
Having learned the authoritative nameserver for the domain, try querying that nameserver directly:
AUTHORITY=(fill in response from the dig ns here)
dig @$AUTHORITY $QUERY
dig ns1.dreamhost.com gothamgazette.net
does it match the answers from the known good DNS server? does it not match the answers from the suspect server?
pay attention to the TTLs of each response, also. It’s entirely possible that the suspect server legitimately cached an old value, and it is just waiting for the TTL to expire. Do subsequent requests to the suspect server decrement the TTL properly? The value should be in seconds.
Try the suspect server again when its published TTL has expired. Does it still give the bad answer? If it does, you might want to look up who controls the nameserver, either by PTR record:
dig -x $SUSPECT
or by whois allocation record:
whois $SUSPECT
If the nameserver is giving out bad information, it might be worth telling the tech or administrative contact of the controlling organization that they’re offering substandard DNS service.
It’s also worth publishing a note about in a blog (or somewhere similar) with concrete details so that other people who find they have similar trouble with the same providers can be assured that they aren’t crazy.
Note that your nameserver might be run locally, possibly in a NAT’ed RFC 1918 address. In that case, you should contact your local adminstrator about the problem. If you *are* your local administrator I’m not. Whew!, you should either repair the nameserver, or fix your DHCP service to suggest a functional nameserver in the DHCP Options (option “domain-name-servers” in ISC’s dhcpd.conf, identified by tag 6, according to the offical DHCP parameters spec [0]).
The nameserver is local, a Windows 2003 server. I’m probably the only one on the network paying close enough attention to be frustrated, but I’ll still post here …
command-line, learning, linux, ubuntu, various
In various on July 13, 2007 at 10:07 pm
Two projects I’m stuck on as Friday draws to a close…
First off, importing the GG site into my new subversion repository craps out when it gets to a particular folder:
Adding /some/path/parks/index.shtml
Adding /some/path/parks/learn.html
Adding /some/path/parks/flash
svn: Valid UTF-8 data
(hex: 2e 5f 50 6c 61 74 65 6c 65 74)
followed by invalid UTF-8 sequence
(hex: 99)
flash is a directory that contains one .fla file, one .swf file and a font directory called Platelet? (yup, with a question mark. I have some ideas about why it is crapping out, but nothing solid. I’d really like me some version control.
[update:] I couldn’t post this without trying a few more tidbits. There was a hidden file (some OSX detritus) that contained an extended character. So that is solved.
Also, (perhaps I’ll fix this before I finish blogging, too) I’m puzzling over the right way to enable remote access to mysql. Along with my fancy pants version control, I’d really like to periodically make a copy of the live database. Or to periodically update my local copy of the database with newer data. Something like that. It is a one way transaction. My options, as I see it, are: mysqldump, mysqlhotcopy and replication
Replication seems excessive, but I could be missing something.
I think hot copy is all I need, but I’m now confronted with the secret truth: I’ve never setup a mysql server before. Not properly anyhow. I don’t know where the settings hide that I need to make mysql accessible over the big bad internets, via a domain name. I don’t know exactly how to do that securely (limit connections by originating IP address?).
I know that I need to change the bind-address setting, but …
* I only want to allow connections from a finite number of IP addresses, and
* I’d like to limit the users who can connect remotely.
[update]: that was actually really easy, too. As it turns out, mysql was already accepting some external connections. The rest is just a matter of GRANT syntax.
Brooklyn, command-line, learning, linux, local, software, technology, ubuntu
In linux on June 15, 2007 at 1:31 am
N’s got a super-sketch-spyware-adware-trojan beast on his computer (that is what you get for browsing with IE, says I) — the kind of thing that you get from clicking the wrong button and launching some kind of ActiveX installer. It won’t go away. No amount of Spybot Search and Destroy or Spyware Doctor will make it go away (speaking of Spyware Doctor, later we can talk about how I feel that the Google Pack is all or nothing. What if I don’t want Google to take over my screen saver? Sure, I can uninstall it later, but why not let me choose in the beginning?).
We’ve given up trying to eradicate it, so I took a field trip to Mikey’s Hook Up where knowledgeable people took a break from their ping-pong game to ask me a few questions and point me towards a reasonably priced external hard drive so we can back up our home drives from his computer and start over. Got that? Knowledgeable people, enjoying their workday. I can’t tell J&R and Best Buy apart, but I am pretty sure that no one really tries to help you at either one. They sure don’t at Radio Shack. And you don’t get to play ping pong on your breaks at Best Buy. It is okay to shop someplace where the staff aren’t being forcibly crushed into a homogeneous model of “Perfect Customer Service” that involves no service whatsoever.
Back to the hard drive, though. I know that we could bring the malware with us, but, having evaluated the risk, we’re not that stressed out about it. Moreover, since the option is to just erase everything, we may as well try.
Since our fancy new LaCie (designed by Porsche, ooh la lah) drive has a gazillion gigs available, I get to back Brahms up, too.
Here’s what I did. I’d love to know if you’ve got a better idea:
[0 amanda@brahms ~]$ cd /media/LACIE/Brahms/
[0 amanda@brahms Brahms]$ sudo tar cvpfz backup.tgz --exclude=/proc --exclude=/lost+found --exclude=/mnt --exclude=/sys --exclude=/dev --exclude=/media --exclude=.Trash /
Note that Ubuntu mounted the drive of its own accord. It is not inconcievable that a person could have to mount a drive themselves if it doesn’t just automagically appear. If you don’t like my only relevant archive, you could try searching the internets.
I made up the “.Trash” part because I couldn’t find any documentation of how the tar --exclude option handles regular expressions. If I backed up the trash, okay, I backed up the trash. Worse things have happened.
Still, I’m wondering. Did I do the right thing? I created separate (duplicate) backups of a couple of directories where I store working documents (my home directory and /var/www/html but those were a little more straightforward.
PS, can i just tell you how much it warms my heart that Princess Oh-my-god-she-saw-me-without-makeup Melissa also is totally over ga-ga weddings. It isn’t just because I’m a difficult and belligerent punk, it is because people go out of their heads and they need to just not.
command-line, learning, technology, various
In various on May 24, 2007 at 3:55 pm
I’m taking a break from a puzzle I really don’t need to be spending this much time on. Here goes:
I heard a rumor that microformats are cool, and, I want to links to a calendar. Seems simple, and a good place to start understanding RDF. Here’s what I’ve got: the original calendar, which basically works for the time being; a script that pulls the links to the calendar (some of it, anyhow) into well formatted RDF (you’ll want to view the source, no doubt).
So, um. What do I do with that? I’ve been playing with some PHP scripts that will parse the RDF six ways to Shaolin but won’t just display anything I can display as html in the <a href=”URL“>Event Title</a> format.
I can generate a huge multi-dimensional array that doesn’t lump items together at all. I can generate a smaller array that only has the Titles and URLs, but they’re not distinguished in the array.
I need to sleep on it, because I think I’m missing something, either about RDF and parsing it or about PHP and arrays. Since I’m a long way from actual sleep, I’m hoping that you, dear reader, will chime in with some brilliant insights (a code snippet or two, perhaps?)
In the meantime, I did come up with a not altogether inelegant solution: I rewrote the script (the one that was generating well formatted XML) and now it generates a nice little HTML snippet. Now if anyone could tell me why this: my $feat_start = POSIX::strftime('%d,%m',@feat_field[0]); doesn’t work, or how this 2454232 is May 11, 2007 if it isn’t a date that strftime can work with … I’d be much obliged.
command-line, learning, linux, ubuntu, various
In various on May 16, 2007 at 4:07 pm
Once, these links helped.
windows clients can’t see workgroup (a post I made to a samba list) and Chapter 38. The Samba Checklist; Part V. Troubleshooting (a diagnostic walk through).
If you’re setting up a SMB network and hitting walls, start with Chapter 38, Part V.
command-line, drupal, learning, technology, various
In various on May 14, 2007 at 1:49 pm
I’m goofing off with Contemplate, trying to play with displaying taxonomy terms in smarter and more structured ways. I’ve gotten as far as sorting out that (correct my syntax on this one, please!) Drupal node taxonomy is stored as a recursive array of objects. Here is one example: Read the rest of this entry »
command-line, learning, linux, technology, ubuntu
In various on May 10, 2007 at 7:09 pm
Used to be, I worked for Mainichi Shimbun, where I learned odd things, including that the Japanese term for “desktop” in the digital sense is “deskutoppo.”
I also don’t really understand why my computer crashes so much and I wish I knew more about command line WPA. It seems wrong to install things willy nilly. Read the rest of this entry »
command-line, learning, linux, technology, ubuntu
In various on April 23, 2007 at 2:13 am
Used to be, I’d moan and groan a lot about how nothing since HomeSite has had a halfway decent regular expression editor. Made me cuckoo. Quanta only handles one line at a time, and don’t get me started about KFileReplace or I might have to tell you about how the helpfiles for regular expressions in KFileReplace, rather than actually explaining their RegEx syntax, basically just says that you shouldn’t use regular expressions. Wha?
But today, I learned about :arg and :argdo and I’m in vim heaven. See for yourself: http://www.vim.org/htmldoc/usr_26.html
[tags]ubuntu, linux, command-line, learning, technology[/tags]
command-line, learning, linux, technology, ubuntu
In various on March 16, 2007 at 3:10 pm
Used to be, I had a Mac. I complained some, I liked it some. Whoopee.
One thing it did that I liked, was it streamlined local name based virtual hosting. If you do any kind of web development, that can be pretty helpful, name based virtual hosts on your local computer. Now that I don’t have a Mac anymore, I have to do these things for myself.
Without a super full understanding of how Apache interacts with, um, other stuff, I got about as far as making a virtual host entry, knowing full well that I was missing something but not having the first idea what I was missing.
I finally buckled and talked to dkg who, of course, explained the fifteen different ways that I could get name based virtual hosts working locally. I opted for the /etc/hosts option, and added a line to my hosts file that maps “http://thisproject.local” to the loop back IP address 127.0.1.2.
The text of each file is below, but I’m still secretly stumped. I’m missing something about how this all works, as I never defined that IP address anyplace else, and while http://thisproject.local works just fine, http://127.0.1.2 still maps straight to /var/www/index.php.
Here is my /etc/hosts file (part of it, anyhow)
127.0.0.1 localhost brahms
127.0.1.1 brahms
127.0.1.2 thisproject.local
And here is the Virtual Host definition that I enabled (note that I store the actual website files in a way that I find useful, first by project, then by URL. Some projects have a few URLs.)
NameVirtualHost *
<virtualhost *>
ServerAdmin webmaster@localhost
ServerName thisproject.local
DocumentRoot /var/www/thisproject/www.thisproject.com
<directory /var/www/thisproject>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</directory>
ErrorLog /var/log/apache2/thisproject_error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog /var/log/apache2/thisproject_access.log combined
ServerSignature On
</virtualhost>
I’d love some help figuring out how to make this entry more accurate. I’m definitely still missing some things, but I like to writeup what I get working.
(While we’re on the subject of puzzles, Ultimate Tag Warrior seems to be choking somehow. Not sure what the deal is, but I did tag this article with “command-line, technology, learning, linux, ubuntu” and I bet those tags are going to get swallowed. I’ve upgraded WordPress a few times, so I suspect I broke something. )
command-line, learning, linux, technology
In various on March 14, 2007 at 3:48 pm
Continuing with my purely hypothetical project, taking over a server that I didn’t build …
I’m wondering about Yum. Say you’ve got a RHEL4 server, and there aren’t any functional package managers running (up2date is there, but that only looks for kernel upgrades) and you want some package management. I installed Yum from RPMs, but now that it is installed, I don’t have any kind of handle on who to trust for packages. There is no debian stable for RedHat.
I found good instructions on adding repositories to yum.comf, but no reliable advice on which repositories to add.
command-line, learning, technology
In various on March 14, 2007 at 3:35 pm
Say you’re responsible for a server, and you want to take a look at mail that this server is sending out. How do you do that? Hypothetically, lets assume that you didn’t build this server, you just took it over and are trying to reign in all kinds of ghastly nonsense.
First, you need to know what kind of SMTP server or mail transfer agent (MTA) you are running. Odds are pretty good that it is either Sendmail or Courier. If you really aren’t sure, try searching your running processes for either of those: ps aux | grep "sendmail".
Actually, it isn’t that vital to know, but it is useful information. What you really want to know is, what kind of mail is passing through your system.
If you have webmin, it is pretty easy to look through your queue. Just sniff around in webmin for something that looks like “Mail Queue” or something. No big deal. Webmin, however, makes you weak. I know you think I’m kidding about this, but I don’t kid.
If you like the command line, or you want to like the command line, try looking up the man pages for mailq. You might also want to take a look at your mail log (mine is in /var/log/maillog). You’d be amazed at what you find.
If you’re really new to the command line, you might want to learn a bit about bash operators to make this process easier.
command-line, learning, linux, technology
In various on January 8, 2007 at 4:24 pm
Or was that Recovery Is Possible? For a bit more on using a live CD to rescue a machine, see my notes from January.
I had this fantasy (oh, what a fantasy) that puzzles like this were behind me, but I found myself, once again, staring at a wackadoo screen that was decidedly not Windows launching. Something about a boot disk and not being able to find one. Read the rest of this entry »
command-line, learning, linux, plone, teaching
In various on November 27, 2006 at 6:46 pm
I have good reasons for making this move and I don’t want to get pounced on for my decision, so let us just start there. My reasons include hosting costs and the fact that our Plone host hasn’t been a good fit. I recently completed the process of migrating Recycle-A-Bicycle to a new web host and CMS. I moved almost all of our content from Plone into Drupal, and learned a few things along the way. Read the rest of this entry »
command-line, learning, technology, ubuntu
In various on June 30, 2006 at 3:50 pm
Sorry dad, this one is definitely for the what the hell is she talking about? files. Maybe later I’ll write about sleeping under the cowitch tree and Frank Quan’s coffee and waffles and the Grace Quan, and that will be interesting. But for now, I give you …
Ubuntu Networking.
If you want to be able to use the internet and you aren’t a command line cowboy, these commands will help you find your way, since most Ubuntu How Tos expect you to know what kind of modem you’ve got before you set out.
lspci will list all PCI devices. Your internal modem and wireless card are probably still PCI devices. I had to scan the output for 802.11 because there was no mention of WLAN or wireless in the output.
You can also look under system > administration > device manager for a list of known devices on your system. Now that I know I’ve got an Atheros AR5212, I can actually get some useful information from the Network Manager hardware list
Most instructions aimed at making Network Manager work on Ubuntu instruct you to restart all the time. Since some instructions don’t actually work, you may find you’ve restarted for naught. Restarting takes a while, and if you happen to be, oh, doing anything else, you lose it all. And so I bring you sudo /etc/init.d/dbus restart: you can restart networking without restarting Ubuntu.
For what it is worth, what did finally work was to comment out most lines, ‘cept these:
# The loopback network interface
auto lo
iface lo inet loopback
in /etc/network/interfaces and then to restart. Then my wireless interface appeared in the Network Manager panel applet.
Next Challenge: dial up modems, pppconfig, wvdialconfig and more.
And then? Why did Knotes start leaving its Notifications Area icon floating on the desktop rather than docked in the Panel Notification Area? And how do I put it back?
Here is the deal: Knotes launches on startup. I like it, Gnome Stickes stink. For a while, it would launch and put its icon up with the other icons normally. Then one day the icon was floating off to the left and over the desktop. I quit Knotes and relaunched it and the icon went back where it belongs, but ever since then, when I startup the icon is floating in a very weird place on the desktop and not in the panel where it belongs.
PS, Dad, if you can figure Flickr out, you can upload somephotos of China Camp. That would be useful, don’t you think?
command-line, learning, linux, technology
In learning, linux, various on February 27, 2006 at 8:07 pm
I just realized that Jack was looking around for this, which isn’t live yet. Wasn’t. Is now or you wouldn’t be reading it …
Got a Linux server but you don’t know your password? It happens. Usually it happens because someone set it up two years ago and since then it has just worked.
And then one day, for some reason, you couldn’t connect to the file share. One thing you can do is try booting with a live CD but if you just need to reset the password, you’ll need to first boot into single user mode. Because Linux is Linux, nothing is ever simple, starting with the fact that just knowing that you are running Linux doesn’t tell me what boot loader you are running. The boot loader is the program that loads when your computer starts up–this program, in turn, loads your operating system (er. actually it loads the kernel, which loads the operating system, but I don’t want to get ahead of myself). Windows comes with a boot loader, the Mac OS comes with a boot loader. Linux comes with a choice. The odds are good that your boot loader is either Lilo or Grub.
Briefly, from there, here is how you do it if your boot loader be grub and if your boot loader be lilo this’ll help, though I think that’s got directions for Grub, too.
command-line, learning, linux, technology, ubuntu
In learning, linux, various on January 19, 2006 at 2:01 pm
File under famous last words … sure, boot it from CD. Great. And then? What?
This, is what:
http://arainyday.se/notebook/windowsrescue.php
http://www.ubuntuguide.org/
More specifically:
The live CD boots its own operating system. If you want to access files on an existing hard drive, you need to find and mount that hard drive. fdisk -l will show you any attached disks (such as an internal hard drive) that can be mounted. sudo mkdir /media/windows will create a mount point or a place to find the hard drive in your linux file system and then sudo mount /dev/hda1 /media/windows/ -t ntfs -o nls=utf8,umask=0222 will actually mount the drive to be readable in that directory. If your old hard drive is FAT not NTFS, you’ll need a different command: sudo mount /dev/hda1 /media/windows/ -t vfat -o iocharset=utf8,umask=000 (fdisk -l should tell you how the drive is formatted)
Now you have to decide whether you are going to start by backing everything up. A bit of advice if you are actually trying to restore a fried system: you want to start by backing everything up. So, to back things up you need some kind of external drive. Probably a USB type of something, which you’ll need this handy command for: lsusb.
Which tells you that it can see that USB DVD-RW, but doesn’t tell you how to mount and write to it. I got tired of trying to figure that out, though I’m sure that some sequence of four commands would have gotten me up and running toot sweet.
Instead I went back over to the spanking new eMachine they’d bought, enabled a share drive (actually someone else had already done that part) and looked up its IP address (open a DOS prompt and give the ifconfig command). Then I enabled the ethernet interface (system > preferences > networking) and connected to it as a windows share (places > connect to server ... ; the service type is windows share, use the IP address of the destination computer in the server field.) and started copying things over and burning them to CD from the working computer. Not ideal since some kind of trojan horse is likely what brought down the old machine, but it works, and the new computer doesn’t have anything on it yet so this was safe enough.
Question for the kids at home: how could I have actually mounted the DVD writer?
command-line, learning
In learning, linux on November 2, 2005 at 10:34 am
Tags: learning, command line
This happens a fair amount, in different ways: a text file, when I open it with vim in a shell account, has no line breaks — they are all ^M. This file used to be readable, even in Vim. I don’t know if someone else edited it or what. I don’t know who to blame, or how to keep it from happening again. One way to find out whether someone edited it would be to look in the CVS comment history for the file, but I don’t know how to do that, either.
- What does ^M mean?
it is ASCII character 13, aka CR or ^M–the Mac OS carraige return character. Unix and Linux software tend to favor the line feed charcter, ASCII 10, aka LF or ^J. DOS based stuff (eg Windows) use both, carraige return plus line feed \n\r aka just to be tricky.
- How can I eradicate it with a search? The string is a single character, I ought to be able to replace it with line breaks, but I don’t know how to type it.
CTRL-m is theoretically how you type ^M, so you can actually drop it into a search and replace statement, but in order to get it to work (and not just execute CTRL-m) I had to type CTRL-vCTRL-m. Try it out. This command :%s/CTRL-v CTRL-m//g worked for me in vim, on a file with good linebreaks AND loose ^M characters at the end of each line. The CTRL-v part is terminal specific, so you might find that you have to fuss to get the right combo for your OS, terminal and shell.
- How did it get there? Did I (or someone else?) edit the file and inadvertently change the line break formatting?
That, I still don’t quite know. Coulda been a lot of ways.
- And then, how can I see a history of comments or figure out when this ^M thing was introduced? I don’t see anything obvious in the CVS Man pages for History.
That would be because history is the wrong command. Try cvs log index.php to see the list of comments and cvs diff -u -r 1.3 -r 1.4 index.php to diff to recent revisions (in this case 1.3 and 1.4).
How do I solve this problem?
These commands are important but they don’t actually work in every case:
:set ff? will tell you what vim thinks the file format is. You can change it with these:
:set fileformat=unix
:set fileformat=dos
:set fileformat=mac
You can search and replace within vim:
:%s/CTRL-vCTRL-m/CTRL-vCTRL-m/g which will come out looking like :%s/^M/^M/g as you type it. It seems a little backasswards that this would work, but it happens to. If that offends you …
You can use the “translate” command, tr, which can do character substitution from outside of vim:
tr '\r' '\n' <macfile.txt >unixfile.txt which will replace mac CR characters in macfile.txt with LF and save it as unixfile.txt
tr '\r' '\n' <macfile.txt | grep word which will allow you to search the file for lines containing word– grep doesn’t recognize CR line breaks so it thinks the whole file is one line, which doesn’t yield very helpful line breaks. This one won’t change the macfile.txt, but grep will search the output.
Or a perl script:
perl -p -e 's/\r/\n/g' macfile.txt >unixfile.txt which will convert macfile.txt to unix format, and save the result as unixfile.txt. This is functionally identical to the first example, but since perl is actually a very general programming language, it can also do some other useful things… BTW, he -e means the program will be the next thing on the command line (‘s/\r/\n/g’ – perlese for replace all \r’s with \n’s), and the -p means do this for each line of the file.
perl -pi -e 's/\r/\n/g' textfile.txt will convert the file textfile.txt from Mac-style (CR) line breaks to unix-style (LF), and replace the original file with the converted version (that’s what the -i means).
perl -pi -e 's/\r\n?/\n/g' textfile.txt convert the file textfile.txt from Mac-style (CR) or PC-style (CRLF) line breaks to unix-style (LF), and replace the original file.
perl -pi -e 's/\r\n?/\n/g' *.txt convert all text files (or rather, files with .txt extensions) in the current directory to unix-style breaks. Note that any that were already in unix format will not be changed.
perl -pi -e 's/\n/\r/g' textfile.txt convert the file textfile.txt from unix-style (LF) line breaks to Mac-style (CR), and replace the original file.
I think you can do much of the same with sed, but I don’t know sed so well.
Gratitude: these guys helped me figure out how to say this all coherently. As did the Vim tipsheets on sourceforge