I spent a whole day working on a kernel panic issue. I tried recompiling the kernel about a hundred times trying all sorts of combinations of drivers trying to get past this VFS thing. As it turns out, I was using "sda" and should have been using "hda" names instead.
This was very frustrating as the boot CD used the SD names for my drives, but I knew I was running IDE. As least I have this server up and running finally.
Monday, September 29, 2008
Wednesday, September 24, 2008
Rediscovering HSX
I had an account over at http://hsx.com for a long time. (Hollywood Stock Exchange). Somehow this came up at work and my coworker was interested in looking at HSX. That got me to look at my old account, and now I'm going to be doing some major portfolio work during free commission Saturday.
On HSX, you pretty much buy and sell "movie" stocks. Movie stars are bonds and it's really a lot of fun looking for the movies that are going to do really well.
On HSX, you pretty much buy and sell "movie" stocks. Movie stars are bonds and it's really a lot of fun looking for the movies that are going to do really well.
Tuesday, September 23, 2008
The other case for virtualization
We all know that by virtualizing our servers, we reduce hardware and power costs. When looking up information about server virtualization, that's 99% of the reason to do it. There are other and even better reasons, from the stand point of a system administrator, to virtualize.
Where I work, we have many servers that we put into virtual environments. We have a pretty good balance between virtual and iron. For some tasks, people are just not comfortable using a virtual environment like Virtual Center for things like Nagios (monitoring) or Cacti (metrics), so these servers get the bare metal. The IO some servers consume make sharing hardware just not worth it.
Now even though some servers deserve their own hardware, they can still be virtual servers on that hardware. Even if it's a single virtual server on a single hardware platform (though running two would be nice so you can have a fail-over cluster).
Portability - If all servers are virtual, even the single ones (one virtual to one hardware platform), then upgrading hardware is painless. The server and hardware are no longer attached so just about anything with room and power can house this server.
Server builds - all of the hardware can now be installed using pretty much the same template. All you do is rack up hardware and build it as a dom0 (or host). Then you can just copy or build the domU (guest) to the new hardware.
Remote Console - With virtual servers, you can now just ssh to dom0 and then bring the virtual servers to single user mode, to do maintenance.
Lights Out Management - Along with the remote console, you also gain the ability to remotely power off the server or reboot in case of kernel panic. This is fantastic if you have remote sites.
Redundancy (with clustering) - Redundant power supplies, dual NICs, RAID... We put a lot of money into making servers safe from hardware failure, and maybe because of this I don't really ever have a server die on me due to hardware. Oh, I have to replace hard drives frequently, but I never had that drop a server on me. What I -do- run into are kernel panics killing the server. In fact, most of my server downtime is caused by software issues.
Cheap STONITH - If you cluster two together on the same system, you can "Shoot The Other Node In The Head" via dom0 so you save on having to buy special power strips.
I know that this makes the server more complex, and you can also run into problems on the base OS, but if you really tailor that build to be dedicated to just running virtual servers, you really minimize that risk. It's really the same risk as just running the one OS for the one hardware platform.
Where I work, we have many servers that we put into virtual environments. We have a pretty good balance between virtual and iron. For some tasks, people are just not comfortable using a virtual environment like Virtual Center for things like Nagios (monitoring) or Cacti (metrics), so these servers get the bare metal. The IO some servers consume make sharing hardware just not worth it.
Now even though some servers deserve their own hardware, they can still be virtual servers on that hardware. Even if it's a single virtual server on a single hardware platform (though running two would be nice so you can have a fail-over cluster).
Portability - If all servers are virtual, even the single ones (one virtual to one hardware platform), then upgrading hardware is painless. The server and hardware are no longer attached so just about anything with room and power can house this server.
Server builds - all of the hardware can now be installed using pretty much the same template. All you do is rack up hardware and build it as a dom0 (or host). Then you can just copy or build the domU (guest) to the new hardware.
Remote Console - With virtual servers, you can now just ssh to dom0 and then bring the virtual servers to single user mode, to do maintenance.
Lights Out Management - Along with the remote console, you also gain the ability to remotely power off the server or reboot in case of kernel panic. This is fantastic if you have remote sites.
Redundancy (with clustering) - Redundant power supplies, dual NICs, RAID... We put a lot of money into making servers safe from hardware failure, and maybe because of this I don't really ever have a server die on me due to hardware. Oh, I have to replace hard drives frequently, but I never had that drop a server on me. What I -do- run into are kernel panics killing the server. In fact, most of my server downtime is caused by software issues.
Cheap STONITH - If you cluster two together on the same system, you can "Shoot The Other Node In The Head" via dom0 so you save on having to buy special power strips.
I know that this makes the server more complex, and you can also run into problems on the base OS, but if you really tailor that build to be dedicated to just running virtual servers, you really minimize that risk. It's really the same risk as just running the one OS for the one hardware platform.
Monday, September 22, 2008
Sick people at work
There is a sick war in the next cube row. Two windows admins are sick
and a third is trying to defend his health. One of the sick admins
got sprayed in the face with Lysol. If there is a coughing fit, soon
after I can smell a blast of Lysol drifting over the cubes.
and a third is trying to defend his health. One of the sick admins
got sprayed in the face with Lysol. If there is a coughing fit, soon
after I can smell a blast of Lysol drifting over the cubes.
Friday, September 19, 2008
Sending shell commands to Syslog
I have been interested in tracking all shell commands on my servers as well as putting something together for my "hacker terrarium". The general consensus is that you need to tweak the source code for the shell and recompile it. That might be the "cleanest" way to do it, but it's just not the easiest. It's best to just have a simple change that I can apply to hundreds of servers.
I came across a blog entry that had this gem buried in a diff file. All you really need to do is to add a function to /etc/profile, and you're done. Here is the snip:
function history_to_syslog
{
declare cmd
cmd=$(fc -ln -0)
logger -p local7.notice — SESSION = $$, CMD =$cmd
}
trap history_to_syslog DEBUG
The big part to that function is the "fc" (fix command) built-in command to bash. The -l option send the command to standard out and the -n suppresses command numbers. Check "man bash" for more details and options.
Once that function is in /etc/profile, fire up a "tail -f /var/log/messages" then start another session and test it out. You'll notice that shell commands get sent to syslog, but it has a delay of one command.
It's not as clean as modifying the source, but it does get the job done and should satisfy a lot of audit requirements.
Source of info:
http://posludio.wordpress.com/2007/11/02/bash-history-to-a-remote-syslog/
Talk like a pirate day
[http://en.wikipedia.org/wiki/International_Talk_Like_a_Pirate_Day]
So a pirate walks into a bar with a steering wheel attached to his crotch. The bartender says "Hey! You have a steering wheel attached to your crotch!" and the pirate says "Arrr! And it's driving me nuts!"
So a pirate walks into a bar with a steering wheel attached to his crotch. The bartender says "Hey! You have a steering wheel attached to your crotch!" and the pirate says "Arrr! And it's driving me nuts!"
Thursday, September 18, 2008
Project: Hacker Terrarium
So I built a firewall, and that was fun watching the denied traffic. The I added portsentry to have a little more fun with the SSH and Telnet ports. Now I want to see what do these people plan on doing if they got access to my server?
I decided to build a honey pot, or honey net. I'm planning on building a xen server and use the virtual guest (or DomU I guess) as the sacrifice. I'm still in the building of the server phase and I'm using Jeos (Just Enough OS) [http://www.ubuntu.com/products/whatisubuntu/serveredition/jeos] however I wonder if just a nice minimal Debian build would have been better.
I want to find out if I can build a kind of snapshot xen guest. One that every time I reboot it, it will return to it's pristine state so that no matter how much a script kiddie horks the virtual machine, it will always return to a fresh usable state. If the virtual environment doesn't work, I'll have to build a custom live CD. Then I can just keep rebooting clean.
I'd also like to try this with OpenSolaris [http://opensolaris.org/os/]with containers. Maybe I'll try this with all three methods and report the results.
I decided to build a honey pot, or honey net. I'm planning on building a xen server and use the virtual guest (or DomU I guess) as the sacrifice. I'm still in the building of the server phase and I'm using Jeos (Just Enough OS) [http://www.ubuntu.com/products/whatisubuntu/serveredition/jeos] however I wonder if just a nice minimal Debian build would have been better.
I want to find out if I can build a kind of snapshot xen guest. One that every time I reboot it, it will return to it's pristine state so that no matter how much a script kiddie horks the virtual machine, it will always return to a fresh usable state. If the virtual environment doesn't work, I'll have to build a custom live CD. Then I can just keep rebooting clean.
I'd also like to try this with OpenSolaris [http://opensolaris.org/os/]with containers. Maybe I'll try this with all three methods and report the results.
MySQL Password change (Forgot root?)
"Anyone know the password to MySQL on sqlsrvr26?"
For whatever reason, maybe you are starting a job and have to clean up after the old admin left, you may need to reset the root password for MySQL. It doesn't get simpler than this:
For whatever reason, maybe you are starting a job and have to clean up after the old admin left, you may need to reset the root password for MySQL. It doesn't get simpler than this:
/etc/init.d/mysqld stopThe two update commands change root for localhost, and if you need to access it remotely like from PHPMyAdmin, the second update should take care of that. You probably only need the first update command though.
/usr/bin/mysqld_safe --skip-grant-tables &
mysql -h localhost
use mysql
update user set password = password('newpassword') where user = 'root' and host='localhost';
update user set password = password('newpassword') where user = 'root' and host='%';
quit
/etc/init.d/mysqld restart
Wednesday, September 17, 2008
Portsentry history file
I didn't know that I had my firewall running for such a long time. I discovered my portsentry.history file and found it going back to November 2005. I did a quick analysis of the file and came up with some numbers. (firewall + portsentry doc: http://docs.google.com/Doc?id=d6hvqjv_55gq83jz)
I didn't go into very deep detail with the stats, like pie charts and what not. I just used grep with some regex.
Blocked hosts by year.
2006 - 1077
2007 - 427
2008 - 675
Blocked hosts by port.
SSH - 1348
Telnet - 999
When setting up Portsentry, I was making the assumption that port scans would start at a low port number and work their way up the ports. While there are more SSH connections, there are still quite a good number of Telnet probes. Two possible reasons are either random ports (not sequential like I anticipated), or distributed port scans. The idea of a distributed scan is really cool because even if I block a host on the SSH port, the attacker can continue probing for open ports from other hosts. It's nearly impossible to prevent.
Something that I found surprising was the probes by country. I was positive that China was going to rank number one. That and maybe Russia. It turnes out that the top three were Brazil, Italy, and France.
Blocked hosts by country.
BR - 47
IT - 33
FR - 33
JP - 32
DE - 31
MX - 25
CO - 22
TW - 21
CN - 18
RU - 16
China and Russia came in much further down the list than I originally anticipated.
I did a basic grep for hosts that start with mail, www, or ns to get a sample of likely hacked mail, web and dns servers.
Mail Servers - 52
Web Servers - 22
DNS Servers - 16
The mail servers took the cake.
Finally, a look a ISPs
Comcast: 75
Road Runner: 40
Wanadoo: 21
Charter: 17
Shaw: 14
Another shocker for me was Wanadoo, which is a French ISP.
Anyway, I had some fun looking at the logs and thought I'd share. It's only a sample, and most of the hosts in the log are just IP addresses that don't resolve so they were pretty much useless.
I didn't go into very deep detail with the stats, like pie charts and what not. I just used grep with some regex.
Blocked hosts by year.
2006 - 1077
2007 - 427
2008 - 675
Blocked hosts by port.
SSH - 1348
Telnet - 999
When setting up Portsentry, I was making the assumption that port scans would start at a low port number and work their way up the ports. While there are more SSH connections, there are still quite a good number of Telnet probes. Two possible reasons are either random ports (not sequential like I anticipated), or distributed port scans. The idea of a distributed scan is really cool because even if I block a host on the SSH port, the attacker can continue probing for open ports from other hosts. It's nearly impossible to prevent.
Something that I found surprising was the probes by country. I was positive that China was going to rank number one. That and maybe Russia. It turnes out that the top three were Brazil, Italy, and France.
Blocked hosts by country.
BR - 47
IT - 33
FR - 33
JP - 32
DE - 31
MX - 25
CO - 22
TW - 21
CN - 18
RU - 16
China and Russia came in much further down the list than I originally anticipated.
I did a basic grep for hosts that start with mail, www, or ns to get a sample of likely hacked mail, web and dns servers.
Mail Servers - 52
Web Servers - 22
DNS Servers - 16
The mail servers took the cake.
Finally, a look a ISPs
Comcast: 75
Road Runner: 40
Wanadoo: 21
Charter: 17
Shaw: 14
Another shocker for me was Wanadoo, which is a French ISP.
Anyway, I had some fun looking at the logs and thought I'd share. It's only a sample, and most of the hosts in the log are just IP addresses that don't resolve so they were pretty much useless.
Wednesday, September 10, 2008
Command line VS. GUI (Example)
Now that I have a post titled "Command Line vs. GUI", I feel the need to actually have a comparison. In this case, I’m going to talk about backups. I remember doing restores from GUI systems and the options I had to restore files was a check box next to the file or directory I would like to have restored. This is nice for one or two files, or a whole directory of files. I have run into cases though where a customer requires about 400 files restored. These files tend to have the date in the name to make it easy, but the extensions don’t always match up. In one case you might have two files like:
You can see that if someone needs to have “the holiday weekend” restored, that’s three days of files.
When confronted with a GUI, I have to checkbox each one of these files. Not only that, but I have to scan through all the files in the directory and try to pluck out the ones that need to be restored.
Here is the command line equivalent that I would use
The first command generates the list of files, and the second command restores all the files found in the list.
That's why the windows guys get teased with "click, click, click, click".
Foo.bar.200809101300.txt
Foo.bar.200809101300.bin
You can see that if someone needs to have “the holiday weekend” restored, that’s three days of files.
When confronted with a GUI, I have to checkbox each one of these files. Not only that, but I have to scan through all the files in the directory and try to pluck out the ones that need to be restored.
Here is the command line equivalent that I would use
dsmc q backup -inact /some/directory/active/foo.bar.20080910"*" |grep DEFAULT | awk '{print $7}' > /some/directory/archive/filelist.txt
dsmc rest -ina -filelist=/some/directory/archive/filelist.txt /some/directory/archive
The first command generates the list of files, and the second command restores all the files found in the list.
That's why the windows guys get teased with "click, click, click, click".
Tuesday, September 9, 2008
Mathmaticious by Drew D'Amelia
I had just noticed that YouTube must have start keeping track of the videos that I watch because of this new "recommended" videos thing they have. I guess it's a lot like Amazon's recomendations for merchandise base on what you look at or have purchased in the past. Well, sitting in the list was a video called "Mathmaticious". Ok, I'll bite. What is this about, and up came...
This certainly wasn't an amateur job, what could it be? I looked at the submitters other video's and things started to check out. The other videos were of a much different quality that what's usually found on YouTube. I had to know more. At the end of the video is the persons name that created this and the back story. I was very surprised to find out that Drew did the whole thing himself, except for some of the editing.
The website "All Over Albany" did a piece on Drew. The song was written for a "Pi Day" contest at his school.
I think it's a bit unfortunate that everyone is talking about "The Math Kid", and not about the quality of the video. While the song is about math, I was really impressed with the direction and editing of this video. Drew pays great attention to detail in this and his other video and I am sure we'll see many more fantastic videos from him and his friends. I'm already subscribed to his channel, waiting for more.
To Drew, thank you for restoring my faith in future generations. I thought I lost all hope after seeing things like...
This certainly wasn't an amateur job, what could it be? I looked at the submitters other video's and things started to check out. The other videos were of a much different quality that what's usually found on YouTube. I had to know more. At the end of the video is the persons name that created this and the back story. I was very surprised to find out that Drew did the whole thing himself, except for some of the editing.
The website "All Over Albany" did a piece on Drew. The song was written for a "Pi Day" contest at his school.
I think it's a bit unfortunate that everyone is talking about "The Math Kid", and not about the quality of the video. While the song is about math, I was really impressed with the direction and editing of this video. Drew pays great attention to detail in this and his other video and I am sure we'll see many more fantastic videos from him and his friends. I'm already subscribed to his channel, waiting for more.
To Drew, thank you for restoring my faith in future generations. I thought I lost all hope after seeing things like...
Monday, September 8, 2008
Command line vs GUI
The windows group likes to make fun of some of the Linux commands we run, and we toss GUI jokes "click, click, click, Ok, Apply".
Here is a quick pic to better illustrate the difference between the two.
EDIT: I feel bad that people might have been looking for a real example so I made an entry for that in Command line vs GUI (Example)
I Want!
I think I found my new laptop computer, though it's not really a laptop or a notebook. I guess it falls into the "PDA" class.
I have a couple of friends that have iPhones and those things nearly cover everything I want in a "device". I wanted to get an iPhone and started to figure the math. You have to buy the phone and it sounds like a deal @$199. But you have to commit to a two year contract. Voice, data, and text messages are all separate plans. After all is said and done, the two year deal ends up costing about $2000.
I could buy a MacBook for that price and have plenty of change left over for other fun things. It made me think though, what is it I want in a computing device? What appealed to me about the iPhone? I needed to start asking the reverse question. Not what can this device do for me, but what do I want in a computing device? I made a sort of mental list with obvious things like mobility, wireless, web pages, email, and stuff like that. I also made some more oddball ideas like the ability to have a shell so I can ssh to systems, maybe even VPN ability. It would be nice to be Linux based so I can mess with it.
I found out about a great device, Nokia's 810 Internet Tablet. This is an Ultra Mobile PC that has lots of nice things. The nicest thing I find with this is that it's Linux based with flexibility (not crippled). It uses software from Maemo which is Linux for Nokia's tablet devices. I found out that people use SSH, Cisco VPN, and OpenVPN on this. I watched a demo of the internet browsing, and GPS abilities of this thing.
(Promo Video)
Other n810 related videos
As far as the full size keyboard goes, since it has bluetooth, some interesting options open up.
I don't have the 810 yet or the keyboard, but I'm going to be making my case for Christmas. Cross the fingers!!
I have a couple of friends that have iPhones and those things nearly cover everything I want in a "device". I wanted to get an iPhone and started to figure the math. You have to buy the phone and it sounds like a deal @$199. But you have to commit to a two year contract. Voice, data, and text messages are all separate plans. After all is said and done, the two year deal ends up costing about $2000.
I could buy a MacBook for that price and have plenty of change left over for other fun things. It made me think though, what is it I want in a computing device? What appealed to me about the iPhone? I needed to start asking the reverse question. Not what can this device do for me, but what do I want in a computing device? I made a sort of mental list with obvious things like mobility, wireless, web pages, email, and stuff like that. I also made some more oddball ideas like the ability to have a shell so I can ssh to systems, maybe even VPN ability. It would be nice to be Linux based so I can mess with it.
I found out about a great device, Nokia's 810 Internet Tablet. This is an Ultra Mobile PC that has lots of nice things. The nicest thing I find with this is that it's Linux based with flexibility (not crippled). It uses software from Maemo which is Linux for Nokia's tablet devices. I found out that people use SSH, Cisco VPN, and OpenVPN on this. I watched a demo of the internet browsing, and GPS abilities of this thing.
(Promo Video)
Other n810 related videos
As far as the full size keyboard goes, since it has bluetooth, some interesting options open up.
I don't have the 810 yet or the keyboard, but I'm going to be making my case for Christmas. Cross the fingers!!
Thursday, September 4, 2008
Please make this make sense
FORMER MASSACHUSETTS GOV. MITT ROMNEY: "We need change, all right — change from a liberal Washington to a conservative Washington! We have a prescription for every American who wants change in Washington — throw out the big-government liberals, and elect John McCain and Sarah Palin."Mitt Romney doesn't like big-government? What happened to the government when we had a republican president and a republican house? Why do republicans keep on yapping about the virtues of a smaller government, or talk bad about "big-government liberals"?
Don't get me wrong! I'm all about shrinking the size of government down to near nothing (especially at the federal level), but seriously, SHOW ME THE MONEY! Can someone please give me examples of how Republican leadership (past eight years) has actually reduced spending and/or the size of government? I'm not saying that it hasn't happened, I just haven't heard about it and maybe someone (Nutz?) can just point it out. It's been eight years, where is my smaller government?
If someone does have an answer, please leave out references to liberal anything. I am not a democrat, liberal, or any of that. Right now I'm independent with a heavy Libertarian lean. Short of Anarchy, you don't get more "pro" smaller government than that. Republicans had eight years in office and are asking for four more. Would you keep paying a contractor that kept saying he will build an addition to your home, but hasn't done it for past eight years? Why should I believe these people now?
Subscribe to:
Posts (Atom)
Posts
