Friday, March 27, 2009

Rails GET and POST params precedence

When you have GET and POST vars with the same name, Rails munges them into a single hash key variable into the params hash. The order of precedence varied between my development and acceptance environment. Of course, one should avoid this situation where possible :-)

Wednesday, March 25, 2009

cpu states in top and %st (steal)

top on ec2 machines looks like so
Cpu(s): 19.2%us, 1.0%sy, 0.0%ni, 49.7%id, 0.0%wa, 0.0%hi, 0.0%si, 30.1%st

Most of these fields are fairly well documented:
us -> User
sy -> system
ni -> nice
id -> idle
wa -> iowait
hi -> H/w interrupt requests
si -> S/w interrupt requests

The final field, st, is "extra", implemented on virtualized machines.
The best explanation of it I was able to find was here:
http://www.ibm.com/developerworks/wikis/display/LinuxP/Measuring+stolen+CPU+cycles
%steal
Show the percentage of time spent in involuntary wait by the
virtual CPU or CPUs while the hypervisor was servicing another
virtual processor.

AFAICT, it appears that this doesn't necessarily mean that "some other
user" is stealing cpu resources from you; it could be the servicing of
one of the virtualized resources that is being consumed; if that
utilization is not tied back to your user correctly, it would show up
there.

Tuesday, March 17, 2009

InnoDB can use multiple indices on SELECT

Back in the old days, MySQL could only use a single index for a basic SELECT (each join gets its own index). So a query like this would cause trouble:

SELECT * FROM `users` WHERE (city_location_id IN (...) OR neighborhood_location_id IN (...))

The second where condition will cause a full table scan even though there is an separate index for both city_location_id and neighborhood_location_id.

However, if you're using InnoDB tables, MySQL can optimize for this buy creating a index UNION for you. Read more about it here if interested.

http://dev.mysql.com/doc/refman/5.0/en/index-merge-intersection.html

Password lengths, MSN, and MSN Messenger/Passport

I was receiving this error logging in through openfire's MSN legacy network adapter to an MSN handle.

2009-03-17 07:52:13 UTC ERROR [13095/-611309268] aardvarkec2deve:
Client manager: XMPP Message Error on myhandle@live.com
at msn: () FROM: msn.myserver.com -> TO:
myhandle@myserver.com: The password
you registered with is incorrect. Please re-register with the correct
password.

I'd verified that the password worked by logging in at live.com, so I kept hitting my head against a wall. Finally I tried connecting via an adium client directly to the MSN *messenger* network, which logs in through Microsoft Passport authentication servers. Failure. Finally! Something to work with!

Microsoft appears to have two different maximum lengths on passwords used. When you create a user, you can apparently type however many characters you want into your password (good). However, Microsoft's Password service, which is what authentication of IM users goes through, apparently limits the number of characters you can use (I didn't count what the exact limit was). Once I went in and changed our password to be *shorter* everything worked fine.

So, logging into live.com is not a good test for logging into MSN messenger.

The real source of MSN gateway logout problems

I've been watching them all morning, and they're definitely associated with avatars/buddy images. They come with error lines in the openfire debug logs like "MSN: Got avatar retrieval result: PROTOCOL_ERROR".

When I looked a few months ago, there wasn't any further info from the Krakken folks (they make the legacy network gateway plugin for openfire). Now though we have this good writeup, confirming that it is indeed:

1) something flakey going on with the avatar protocol with MSN and

2) that the MSN api has changed a lot since they last maintained it.

http://kraken.blathersource.org/node/34

I'm more and more convinced that using Krakken for our IM network connectivity on legacy networks isn't going to keep pace and we'd be better off switching to libpurple.

Monday, March 16, 2009

rake invoke versus execute: a huge difference

Those more familiar with the intricacies of rake might already know this, but Rake::Task['taskname'].invoke and Rake::Task['taskname'].execute are very very different in how they behave w.r.t. dependencies and chaining.

First, here's a background example explaining how dependencies aren't run when using execute, but are always run when using invoke.

http://chrisroos.co.uk/blog/2007-12-06-ruby-rake-invoke-vs-execute

I additionally discovered that a task called using invoke will only run the *first* time you invoke it, not a second, third, or fourth time.

In my particular case, I wrote a rake task to stop and start mysql on mac osx, and then used it within a number of other tasks. Each task, when I tested it individually ran fine, but when I tried to chain the series of tasks together, everything went horribly awry, and it looked as if mysql wasn't properly restarting. I went digging for why, and discovered that because I was using invoke on my task, the mysql stop and mysql start task was only running when I called it from the first of the chained tasks. Switching to execute resolved this problem.

Here's the example that demonstrates; note that the code for both invoke and execute tests try to run the stop/start sequence twice (stop/start/stop/start). In practice though, the first test with invoke only runs the stop/start sequence once, while the second with execute runs the stop/start sequence twice.


desc 'stop mysql on macosx, requires sudo'
task :stop_mysql_macosx do
puts "Attempting to stop mysql..."
`sudo /Library/StartupItems/MySQLCOM/MySQLCOM stop`
sleep 2
mysql_process=`ps aux | grep mysql | egrep -v "grep|rake"`
raise "mysql did not stop properly, got mysql_process #{mysql_process}" if !(mysql_process == "")
puts "Successfully stopped mysql."
end

desc 'start mysql on macosx, requires sudo'
task :start_mysql_macosx do
puts "Attempting to start mysql..."
`sudo /Library/StartupItems/MySQLCOM/MySQLCOM start`
sleep 2
mysql_process=`ps aux | grep mysql | egrep -v "grep|rake"`
raise "mysql did not start properly" if (mysql_process == "")
puts "Successfully started mysql."
end

desc 'test start/stop mysql macosx using invoke'
task :test_start_stop_mysql_macosx_using_invoke do
Rake::Task['db:stop_mysql_macosx'].invoke
Rake::Task['db:start_mysql_macosx'].invoke
Rake::Task['db:stop_mysql_macosx'].invoke
Rake::Task['db:start_mysql_macosx'].invoke
end

desc 'test start/stop mysql macosx using execute'
task :test_start_stop_mysql_macosx_using_execute do
Rake::Task['db:stop_mysql_macosx'].execute
Rake::Task['db:start_mysql_macosx'].execute
Rake::Task['db:stop_mysql_macosx'].execute
Rake::Task['db:start_mysql_macosx'].execute
end


~/tmz/aardvark/trunk/dragonfly > sudo rake db:test_start_stop_mysql_macosx_using_invoke
(in /Users/nathan/tmz/aardvark/trunk/dragonfly)
Attempting to stop mysql...
Successfully stopped mysql.
Attempting to start mysql...
Successfully started mysql.
~/tmz/aardvark/trunk/dragonfly > sudo rake db:test_start_stop_mysql_macosx_using_execute
(in /Users/nathan/tmz/aardvark/trunk/dragonfly)
Attempting to stop mysql...
Successfully stopped mysql.
Attempting to start mysql...
Successfully started mysql.
Attempting to stop mysql...
Successfully stopped mysql.
Attempting to start mysql...
Successfully started mysql.

Tuesday, March 10, 2009

i uploaded my xmpp primer to our google docs

Check docs to read if you're interested. It's from this time last year
originally, but it's a great getting started guide.

--
Love,
Fritz

Friday, March 6, 2009

tokyocabinet schemaless database toolset/api

looks like a nice layer around a dbs, following on the pointer in the friendfeed schemaless database writeup. written in C, has api wrapper support for lots of languages, including Ruby.

http://tokyocabinet.sourceforge.net/index.html