User Offline

Sony HX5V camera

2010-04-22T09:13:00.004+01:00

I have been trying to buy a HX5V for the last 2 weeks. None of the main retailers have it in stock and at this rate I might just order one from Tesco (they have one of the cheapest UK prices at a bit under £300) and pick it up when it comes in stock again.

The reason I think this camera is worth it are the basic features of:
* GPS coordinates and compass direction added to photos (in the EXIF data)
* 10x optical zoom
* 200g and pocket-camera sized

It has an interesting variety of other features but GPS is a deal breaker and very few current cameras include it. When I think about our last trip to Southern Greece, it would be handy to have the photos automatically placed on a map as it is easy to forget the name of which village, ruin or beach each photo was taken at. On my T100 we have had some success at taking video so the ability to record in high resolution (AVCHD / 1080i) would be interesting to play around with.

A break with the philosophy of "product-lock-in" by Sony has been to allow the use of SD/SDHC memory cards as well as the Sony PRO-duo. I note there is no upper limit on card memory in the specification, though they do sell 32GB cards to go with the camera which is as large as I could imagine using in the next five years (the largest I own currently is an 8GB card).

KeePassX

2010-04-02T12:32:00.006+01:00

My use of rather feeble passwords has been on my mind of late. Following a recommendation I have tested out the free and open-source KeePassX application. This runs on multiple platforms including OSX, mobile phones or a portable application from a USB stick. It's rather neat to be able to update passwords on my Macmini and then copy the same encrypted password file to my trusty Sony Ericsson K750i for use on the go.

It works as a pretty standard encrypted password wallet but the feature I've enjoyed most is the automatic password generation. This means I can generate random passwords with various options as to whether to include uppercase, lowercase, numbers and odd characters. For most online applications where I might login from someone else's PC I have switched over to using a 16 character random mix of uppercase and numbers on the basis that I might have to type it in by hand. For my various other sites, particularly those that remember my credit card details, I have used 25 character or more random mixes of uppercase, lowercase and numbers. Due to a vague concern about personal data, I have whacked on a human unfriendly 50 character password to my Facebook and Evernote accounts. Okay, in the latter case I could type in the 50 character password by reading it off my mobile phone but I really cannot imagine having to log in to a strange or temporary computer to access these applications.

It seems odd that some websites that store your credit card limit passwords to a crackable 12 characters. I have one such site but I'm using a fully random password with special characters.

As it feels a lot easier not to have to remember passwords in my head, I am now far more likely to change passwords after, say, logging into my Amazon account while on holiday from an internet cafe.

Sites:
http://www.keypass.info
http://www.keypassmobile.com

Selling my second-hand paperbacks and Evernote

2009-08-04T09:51:00.007+01:00

Book photos in Evernote

I have more than 300 of my books up for sale. The vast majority are paperbacks with hardly any value and I list these on GreenMetropolis (if it sells you get £3 per book but have to take the postage out of that). A handful have some second-hand value and I sell the ones worth more than £5 on Amazon (anything less than that and it's not worth the fees to Amazon).

My average is to sell 4 books/month, so I have a fair stock of books to keep tabs on. At the moment there are three places around the house where they are stashed and it was getting quite irritating when one had sold and I was having trouble locating the book (they only grouping I have was splitting fiction from everything else).

Searching for "Kraken"

I stumbled across a fairly easy way of getting around this problem using Evernote. By taking snapping some digital photographs of the stacks of books with the spines showing and then pasting these to Evernote, the online application craftily does its text recognition thing and gives you a fairly good chance of finding the photograph of the book (and hence location) by typing in part of the title. Blam, instant book database.

It's actually an incredibly handy time-saver for double checking where the book is, before going around the house reading every book spine.

Making a Google Calendar link

2009-06-04T18:58:00.005+01:00

There is a handy form available under Google help to create a website button for a calendar event (http://www.google.com/googlecalendar/event_publisher_guide.html). It is pretty useful but I needed to decode the parameters in order to get some javascript to create a particular link on demand. Here's my notes about the parameters you can play with.

anchor address
http://www .google.com/calendar/event?
This is the base of the address before the parameters below.
action
action=TEMPLATE
A default required parameter.
src
Example: src=default%40gmail.com
Format: src=text
This is not covered by Google help but is an optional parameter in order to add an event to a shared calendar rather than a user's default.
text
Example: text=Garden%20Waste%20Collection
Format: text=text
This is a required parameter giving the event title.
dates
Example: dates=20090621T063000Z/20090621T080000Z (i.e. an event on 21 June 2009 from 7.30am to 9.0am British Summer Time (=GMT+1)).
Format: dates=YYYYMMDDToHHMMSSZ/YYYYMMDDToHHMMSSZ
This required parameter gives the start and end dates and times (in Greenwich Mean Time) for the event.
location
Example: location=Home
Format: location=text
The obvious location field.
trp
Example: trp=false
Format: trp=true/false
Show event as busy (true) or available (false)
sprop
Example: sprop=http%3A%2F%2Fwww.me.org
Example: sprop=name:Home%20Page
Format: sprop=website and/or sprop=name:website_name

My experience setting up a 1 Terabyte USB drive on a Macmini

2009-05-19T10:11:00.019+01:00

I already have an external 500GB USB drive but with Leopard time-machine taking half of this, and keeping a backlog of downloaded films to hand with several backups from other machines, this means that it's been running pretty full (in fact only about 10GB general storage left and time-machine backups only run 2 months into the past before being deleted forever).

1TB drives (1,000GB) have come down nicely in price and I managed to buy a Western Digital Elements drive on Amazon for under £80. Price was a significant factor for me, as well as compatibility for re-use with other machines, so I didn't really consider the alternative of a firewire compatible version (it would have been potentially 5x faster than a USB 2.0 connection).

How the 1TB USB drive ("Zeus") appears on my desktop using a Matrix icon

It was pre-formatted but as I'm going to use this one exclusively on Leopard, I decided to reformat with "Mac OS Extended" into two partitions, one with 300GB for the time-machine (backup) and the rest (631GB) for my ever growing document, video and photo archives plus around 80GB of other computer archives (some dating back to the 1990s but you never know when some old document might be interesting to search out).

Step One: Reformat the drive

Easy part; plugged in the drive into a spare USB port and using Leopard's Disk Utility, chose the option for two partitions, Mac OS Extended (not journaled), typed in required partition size and let it go ahead.

Step Two: Transfer existing files

This was more problematic. I wanted to transfer around 200+GB from my old USB drive so I just used drag & drop using the Finder but it turned out that the estimated time to transfer was around several days! A little more research, eventually using the System Profiler showed that the USB port (on a USB extra hub) the new drive was plugged into was rated as USB1.1 speed (why can't the Finder show you this information?) so I had to use one of the ports on the main Macmini box to get it running at USB2.0 speeds (12x faster). This still gave an estimated 12 hours to transfer the files but at least this could run overnight.

Note that the real times for transfer I achieved were nowhere near the theoretical maximum (put 200GB/USB2.0 into WolframAlpha and you get 55 minutes). This was probably due to the old drive being formatted in (non-native version of) NTFS and though I have an open source driver to enable read/write access, it's pretty likely that this means extra processing power is needed to get the data transferred.

It was a good lesson, USB can be slow, so moving large archives is worth a little planning. Obviously on a day-to-day basis the largest things I'll move would be video files but these are on average less than 1GB and it's not an issue to wait for a minute (assuming USB2.0, on a USB1.1 connection, 1GB takes a minimum of 11 minutes to transfer).

WD Elements 1TB USB 2.0

Step Three: Setup Time Machine

I went into Time Machine settings (via System Preferences) and changed the drive to be used to the new blank partition.

[This is the bit I failed to sort out] I wanted to copy across all my previous Time Machine archive files on the old disk into the new folder created on the new disk. This was another 123GB (3½ million items), but after trying to do this several ways (including making both drives read/write after Time Machine made them read-only) I found it invariably caused Finder to lock up. Eventually I gave up and started Time Machine from scratch (it was still an 82GB initial backup).

Conclusion

After all the transfers, and with 300GB reserved in a partition for Time Machine, I now have 380GB spare for new files. As for the old 500GB drive, it's become a nifty media and backup disk shared on my home network connected to a Vista desktop. As this drive had previously been re-partitioned on my Mac, I took the precaution of reformatting it from scratch using Vista to ensure a stable drive standard.

Final speeds: Transferring a 700MB video file to my new archive (named Zeus) now takes me around 15 seconds. Compare that to 2½ minutes to transfer the same file from the old USB drive, across my home network, to get to my new drive. Let's hope USB3.0 gets a move on!

Evernote as my receipt wastebasket

2009-03-01T14:37:00.005Z

Evernote seems to be quite a neat solution to an ongoing problem of what to do with receipts. Most of my receipts end up going in the shredder with various more meaningful ones left stacked on a bill-hook on my desk (like certificates of posting). This is sort of okay but it would be nice to turn this into useful data of what, when and how I bought something. Evernote also ensures that everything gets backed up online and I can then access the information from another location (like being in the office and someone asks "how much did you pay for your TV?").

I use a Fujitsu scanner to keep most of my statements, letters and dreaded tax demands under paperless control and it's a relatively simple thing for me to shove my receipts through it too. One drawback of Evernote (the free version anyway) is that it's a bit crap at handling PDF files. My route for getting receipts onto the Evernote website is to save up a batch of around 30, shove them through the scanner, batch convert pdf to jpg (using Adobe Bridge) and drag & drop them into my Evernote notebook on the desktop application.

Things would be more straight forward if I had a decent webcam on my Macmini, but that's not the case at the moment. The benefit is that I don't have to name the files or organize them in any way as Evernote will recognize text in the image so I can quickly search for "fridge" and the right receipt with its 2 year guarantee from John Lewis will pop up.

As well as archiving away (and shredding most of) my receipts this way, I'm going to do the same for train tickets and film ticket stubs as a useful aid to my poor memory. My trusty Sony Ericsson K750i is capable of taking reasonable photos too, so I might start using it to keep more shots of the type "look for one of these second hand on eBay" sort.

Creating an eBay product turntable

2009-02-25T12:32:00.008Z

Something I have been pondering for a while is a (free) way of creating a rotating product view on eBay. Some professional sellers do it so I knew it was possible. A bit of research today on wiki.panotools.org led me to a way of using a freeware .jar in an applet to do exactly that.

I reused an empty CD spindle box as a turntable (the transparent top makes a nice reflective surface with a depression that a CD can sit in to form a small Lazy Susan). I then took 8 photographs of a mug I could list on eBay and got my first example up within around an hour of messing about. Hopefully now I know how to use Photoshop to create a strip of photos and automate corrections, the next one will be a lot faster. In particular I have a load of matchbox cars to sell and as they are only worth a couple of quid each I needed a fast way of listing them.

Example turntable (16 photographs):

Needs java.

Turntable (use mouse to stop/drag to turn)

Still waiting for my next gadget to be released

2009-02-16T14:48:00.001Z

The economic downturn has made electronics companies draw back from releasing new technology. Here are the key gadgets that I would gladly layout good dosh for but don't think they'll exist in 2009:

Better camera specification

Must - built in GPS to tag photo location in EXIF
Must - 5x or more optical zoom with internal lens
Must - pocket sized
Must - image stabilization
Must - good in low light conditions
Must - take decent video of unlimited length (> SXGA @ 30 fps)
Must - have excellent burst mode (> 7 shots/s)
Must - under £250

Nice - waterproof
Nice - integrated with mobile
Nice - can sync with external flash

Better notepad specification

Must - web tablet touch screen
Must - screen larger than 10cm wide (iPod Touch is less than 9cm)
Must - screen resolution > SXGA
Must - wifi
Must - GPS so I can find my way on foot or by bus
Must - less than £200 (otherwise I'll just get a netbook)
Must - take standard memory cards (like SD or M2)
Must - takes open source software
Must - weigh less than 500g with charger (my laptop weights 1.2 kg and it's a drag)

Nice - expanding keyboard 3/4 size or more
Nice - built in cheap (£2/month) wireless broadband deal

Better usb drive specification

Must - 32GB of storage space (I have 19GB of photographs to carry around)
Must - small (preferably tiny) with intrinsic plug cover
Must - under £30
Could be - a M2 memory card with adaptor (so it can be used in a mobile phone)

Second Life hexagonal prism space station

2009-02-05T17:32:00.009Z

Reusing the modules from my tetrahedron experiment, I fairly easily put together a script to create a hexagonal prism of any size. Again debugging the intelligent floor tiles to automatically detect if they are the floor was a real pain due to the lack of an easy way to compare rotations (rather than vectors).

The one generated in the snapshot is of size 3. The square walls are 30mx30m and the longest floor width is 60m. Total prim count = (walls) 6x3x3 + (roof and floor) 2x6x9 = 162.

Update: I had been using my Sony laptop to work with Second Life. With hubby away I borrowed his PC (I'm normally on a Macmini) and was surprised to find the way SL rendered quite different. In particular the equilateral panels I had been using were set with "glow" at 1.0 (maximum) which made little difference on the laptop but on the desktop were mind bogglingly glowing with green radioactive light. Obviously the laptop graphics doesn't render the glow at all. I ended up muting the glow down to zero as they look pretty bright with the "full light" option on (which makes them ignore local lighting conditions).

Second Life Tetrahedron maker

2009-02-04T13:52:00.008Z

Finally debugged the LSL script to generate tetrahedrons of any size using equilateral triangular panels of edge size 10m. In the picture you can see my avatar inside a tetrahedron of size 2 next to one of size 3 and I'm inside a massive tetrahedron of size 7 in the shot below.

The last formula to work out (which I'd fudged a number for before) was the distance between the avatar (or generating object) and the centre of the tetrahedron side. This is actually N*10m/sqrt(24) where N is the size of tetrahedron.

It took a little while to get the side on the bottom to detect itself and texture itself with grass rather than being glass when rezzed. My problem was that llVecDist() which I was using to compare the rotation of a panel to the vector rotation I was looking for (<pi,0,0> or <3*pi/2,0,pi>) was ignoring the fact these are angles and so I had to get the sign right (so the last vector had -pi in the z-axis rather than pi).

All those glass panels make for a classy looking instant space station at 4,000m.

Now I have the subroutines to create equilateral platforms of any size I might try a future experiment using them to make something like a truncated cubic honeycomb.

Now to do some proper work!

Second Life creation problems with equilateral triangles

2009-02-04T08:07:00.012Z

In Second Life you can create any 3D object using the basic building blocks (called Prims from primatives) like cubes and spheres. These are a maximum size of 10x10x10m (unless you are a God I guess).

I wanted to create the maximum size possible equilateral triangle (in order to help make a giant tetrahedron), unfortunately the "prism" prim was less that 10m on a side maximum. Instead I flattened a 10m cube and tapered it to be a triangle. To turn it into an equilateral I changed the height from 10m to 8.66m, as using Pythagoras' theorem the height must be:
√(10²-5²)m=√(25x(3-1))m=5m√(2)
≈8.660m

Now here's the unexpected problem. When you rotate the large equilateral, it turns out that the centre of gravity is still based on the cube and so has been set to be at half the height. Consequently to rotate correctly about a centre I had to add a correction vector (unfortunately it's not possible to reassign the centre of gravity of a prim).

The centre of an equilateral triangle can be found by bisecting the angles, the resulting 3 central lines intersect at the centre. Looking at the intersection of one of these bisecting lines and the height line gives a right-angled triangle with 30° at the corner and with a base of 5m. Consequently the distance from the base to the centre of gravity must be:
COG=5m*tan(30°)
=5m/(√3)
≈2.887m

So my correction factor to move to the centre (in the Y-plane) is:
CF=[(5m√2)/2]-[5m/(√3)]
=5m(1/√2-1/√3)
≈0.649m

I kind of wish I'd stuck to the prism prim as this correction factor has really sullied the beauty of my LSL scripts ;)

Second Life mathematics

2009-02-01T11:53:00.017Z

About a year or so ago I played around with Second Life but found it was limited as an environment due to lack of integration with IM or Skype (I was looking to use it as an intuitive conferencing tool). It still lacks these features but I've had a little fun recently playing around with the scripting language to play with polygons using Second Life as my geometry lab.

The maximum object size (normally) you can create is a 10x10x10m cube. You can reshape this up in different ways (like a cylinder) but with the premise of making a simple house/room I created a couple of scripts to make giant cubes and domes using a 10x10x0.025m panel as a building block.

My recent experiment was to create a script to use these flat panels to make an N-sided polygon shaped wall in a ring around my avatar. The only tricky bit of math is the distance between the centre point (my avatar) and the 10m wide panel.

This formula does the trick:

distance =

panelwidth ⁄ (2 \times \tan [π / N])

You can work this out by thinking about the right-angled triangle between the avatar, middle of the panel and the edge where the next panel joins:

The base of the triangle shown must be ½ the panel width
the angle "a" cuts up the circle into 2xN triangles (a full circle is 2π in radians) and so "a" = 2π/2N = π/N.
Basic trigonometry gives us the tangent of "a" equal to the opposite side (which we know) over the adjacent side (which is the distance we want).

Programming bit: Note that SL limits the distance you can create (rez) an object to 10m, so my script creates the panel in a small size near the avatar (llRezObject) with the right final rotation, orders the panel (the panel has its own script listening for this command) to the target position (llSetPos) and then the panel resizes itself (llSetScale) when it realizes it has reached the target position (llGetPos). Phew.

The importance of FoxMarks

2009-01-29T12:14:00.004Z

Ask yourself this question, if I lost all my browser bookmarks would I know what I've lost? It's quite amazing how much time most surfers invest in organizing and collecting their favourite places on the internet.

I have recently installed Foxmarks which backup your favourites and allows you to access your bookmarks on the internet. I particularly like the password saving part as this means that I can create new accounts and passwords on websites and log in without any problem on another machine. The profiles feature is pretty neat too as this means I can restrict my "@work" version to professional interest rather than overly personal stuff.

I did however take the precaution of using a pretty secure password for the account. The thought of someone hacking all my passwords at the same time gives me the shivers.

Flickr email checkbox ticker bookmarklet

2009-01-03T09:14:00.009Z

One slightly annoying feature of Flickr is that there is no way of filtering your email. In particular I get a lot of group invites and emails telling me someone added me as a contact. Frequently there is no need for me to open these emails and to delete them means individual checkboxes have to be ticked. Here's a bookmarklet that you can drop into your bookmarks that does exactly that.

Within the javascript there is a "filter" variable that you can set to any text you want (at the moment it matches emails with the words 'Invite to' or 'You are' in the subject). Here's the javascript source:

javascript:(function(){var anchors=document.getElementsByTagName('A'),
checkboxes=document.getElementsByTagName('INPUT'),
filter=/Invite to|You are/, i, j;
for(i=0;i<anchors.length;i++){if(anchors[i].innerHTML.search(filter)>=0){for(j=0;j<checkboxes.length;j++){if(anchors[i].href.slice(-16)==checkboxes[j].name.slice(-16)){checkboxes[j].checked=1
}
}
}
}
})()

Bookmarklet to find cheapest service to call a mobile from a fixed line for the current time and day

2008-12-18T18:12:00.009Z

Callchecker (UK) has a great service to lookup the best 0845/0870 service to call a UK mobile from your fixed line telephone. This varies by weekday, evening or weekend rate. The following bookmarklet looks up the right page based on the time on your computer.

javascript:(function(){
var t='weekday',d=new Date();
if(d.getDay()==0||d.getDay()==6){t='weekend'}else{
if(d.getHours()>17||d.getHours<8){t='evening'}};
window.location='http://callchecker.moneysavingexpert.com/\
ukcallchecker/mobile-numbers/'+t
})()

Here it is as a bookmark that you can test out or drag and drop into your browser Bookmarks.

Bookmarklet to add recycle comment in GreenMetropolis

2008-12-11T10:59:00.015Z

I'm selling a fair number of my old paperbacks on GreenMetropolis.com. When listing a book I invariably want to note that I will send in recycled packaging in the comments field. I was using iMacro to speed this up but I've finally got around to creating a simple bookmarklet to do the same thing. Here it is:


javascript:(
 function(){
   var cmts=document.getElementsByTagName('TEXTAREA');
   for(var ic=0;ic<cmts.length;ic++){
     if(cmts[ic].name=='comment'){
       cmts[ic].value+='I will send with reused or recycled packaging.'
     }
   }
 }
)()

And here it is as a link you can drag and drop into your own bookmarks or try it out by clicking on it here: Add note to TextArea

Reset text area

This would be fairly easy to customize for your own text or to adapt to a different form that you use regularly and get tired of typing in the same old text or if you are familiar with RegEx tweaking the text you already have in the text box. For example changing the text in the text box to Title Case or Shout.

Creating an intelligent shortcut (bookmarklet) in Firefox

2008-12-09T10:16:00.006Z

On of the frequent address shortcuts I use in Firefox is to redirect "w sometext" in the address bar to lookup a term (sometext) in Wikipedia. As I frequently go to my "Watchlist" in Wikipedia I though it would be neat to default to that page if I had not specified a term. A little research and I'd worked out how to get this to work as a bookmarklet. In the address bar any text added to the shortcut is referenced as "%s".

Using this basic structure it would be pretty easy to add a more complex set of conditions to make the same shortcut do many more things, such as looking up a book and working out if the text you put in was an ISBN or a title.

Here's my example formatted so you can read it:

javascript:(
function(){
  if("%s".length==0){
     // Default
    window.location="http://en.wikipedia.org/wiki/Special:Watchlist"
  }else{
     // Wikipedia lookup
    window.location="http://en.wikipedia.org/wiki/%s"
  }
}
)()

And here it is formatted as a link you can drop into your bookmarks, don't forget to edit it so that it has "w" as a keyword/shortcut:
Wikipedia lookup

Moving my mobile number from Virgin to the evil ASDA / Wal*mart empire

2008-12-01T17:33:00.003Z

Sadly Virgin aren't doing all that much to compete with the johnny-cum-lately ASDA. ASDA are currently charging a flat rate for PAYG of 8p/min for all UK landline and all mobile calls with 4p per text message. Virgin claimed to charge 15p/min and 3p for texts on PAYG but in practice (as I don't know anyone using Virgin and my usage is so low it was never worth buying a "bundle") their charges were actually a whopping 40p/min and 12p/text.

It's a great pity but as my average bill even at these rates is always less than £5/month I can't afford to ignore the ASDA deal.

Transfer happened this weekend. I emailed Virgin via their website on Thursday, phoned ASDA with the transfer code on Friday and my mobile number was tranferred today. Surprisingly painless compared to the last time I had to do this (it took 3 weeks).

Use Greasemonkey to link UK postcodes to Google Maps

2008-11-16T10:28:00.007Z

See my last blog post for general details about using Greasemonkey. This titbit of source code uses a large standard regex to match any UK postcode in a web page text (you could be matching paragraph elements) and converts it into a hyperlink to a google map of that area. It assumes that this titbit is inside a script that searches relevant elements of the page and tests them as variable "thisElement".

// Convert postcodes to google map links (z=15 puts names on tube stations)
thisElement.innerHTML=thisElement.innerHTML.replace(/([A-PR-UWYZ0-9][A-HK-Y0-9][AEHMNPRTVXY0-9]?[ABEHMNPRVWXY0-9]? {1,2}[0-9][ABD-HJLN-UW-Z]{2}|GIR 0AA)/,"<a href='http://maps.google.com/maps?f=q&hl=en&geocode=&q=$1&ie=UTF8&z=15&iwloc=addr' target=_blank>$1</a>");

Write your own Greasemonkey script to highlight and hide items on a website

2008-11-14T17:41:00.015Z

Here are some handy code segments that have taken me a little while to work out. Greasemonkey is a useful Firefox plugin with a massive shared library of scripts to do interesting things like adding collapsible folders to your online gmail web page. You can also write your own javascript code for Greasemonkey to apply to web sites you name.

The following code I use for a mystery shopping website I visit regularly but their table of possible assignments available in London can be several pages long. This script helps filter and colour in the table with my preferences (for example out of 80 London assignments I currently have 7 highlighted as suitable).


// ==UserScript==
// @name           Website filter
// @namespace      Filters
// @include        http://mywebsite.com

/* Here I'm searching out items in a table that have been
highlighted in bold already. I'm then searching the bold
text and if it contains "William", I then hide the row
completely. I then go on to pick out items with "London"
in the text in dark green */

var allElements, thisElement;
allElements = document.evaluate(
 '//tr/td/b',
 document,
 null,
 XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
 null);
for (var i = 0; i < allElements.snapshotLength; i++) {
 thisElement = allElements.snapshotItem(i);
   // Hide these rows
 if(thisElement.innerHTML.search(/William/i)>=0) {
   thisElement.parentNode.parentNode.style.display="none";
 }
  // Highlight these elements
 if(thisElement.innerHTML.search(/London/i)>=0) {
   thisElement.style.color="darkgreen";
 }
}

/* In this section I'm searching for any table cell and
checking the text for certain postcodes and addresses.
If there is a match, the cell is highlighted in bold
blue text. */

allElements = document.evaluate(
 '//td',
 document,
 null,
 XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
 null);
for (var i = 0; i < allElements.snapshotLength; i++) {
 thisElement = allElements.snapshotItem(i);
  // Highlight these addresses
 if(thisElement.innerHTML.search(/SE(22|15|1\s)|London Bridge/i)>0) {
   with(thisElement.style){
     color="blue";
     fontWeight="bold";
   }
 }
}

// ==/UserScript==

Explanations

Regular expressions (Regex) can be a bit cryptic but as you'll need to tailor these in the above code I though I'd explain what these bits do:
search(/William/i) ==> Search anywhere in the string for "William" and return the starting position within the string, the 'i' means that it will not be case sensitive.
search(/SE(22|15|1\s)|London Bridge/i) ==> Search anywhere in the string for "SE22", "SE15", "SE1 " or "London Bridge", ignore case and return the position.

You will also note that the first search that hides matching rows uses parentNode twice on the assumption that the bold text will be in a cell which will be in a row, so the second parent node is the "TR" tag containing the row. By changing the style.display attribute this dynamically hides the row from view (to reset and show the row again you would set style.display to null). Phew.

Useful links

Greasemonkey: https://addons.mozilla.org
XPath syntax: http://www.w3schools.com/XPath

iMacro script to swap between Flickr albums

2008-11-10T10:36:00.011Z

The following handy iMacro script logs you from one Flickr account to another. I currently have six of these, one for each separate free Flickr album/account I have on the go (there's a maximum of 200 photos you can share for free in each account). All you have to do is install the iMacro plugin into your browser and save the following script not forgetting to add your login and password in the "User set variables" section.

Obviously if Flickr change their main site layout then this may break the script and it would have to be tweaked.

Script Starts

' Last Edit: 13 Nov 2008

' User set variables
' !VAR1 = your login ID, !VAR2 = your password
SET !VAR1 yourlogin@yahoo.com
SET !VAR2 password

' This script is intended to toggle you out of your
' current Flickr account and into one set by the
' variables above so you can easily swap between
' several albums

' Assumptions
' - A Sign Out link is on every Flickr page
' - You are currently logged into Flickr

' Main script
TAB T=1
FILTER TYPE=IMAGES STATUS=ON
SET !ERRORIGNORE YES
TAG POS=1 TYPE=A ATTR=TXT:Sign<SP>Out
TAG POS=1 TYPE=A ATTR=TXT:Sign<SP>in*
WAIT SECONDS=2
TAG POS=1 TYPE=A ATTR=TXT:Sign<SP>in<SP>as<SP>a*
SET !ERRORIGNORE NO
WAIT SECONDS=2
TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:login_form ATTR=NAME:login CONTENT={{!VAR1}}
SET !ENCRYPTION NO
TAG POS=1 TYPE=INPUT:PASSWORD ATTR=ID:passwd CONTENT={{!VAR2}}
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=VALUE:Sign<SP>In
FILTER TYPE=IMAGES STATUS=OFF
URL GOTO=http://www.flickr.com/activity
TAG POS=1 TYPE=SELECT FORM=* ATTR=ID:act-since CONTENT=%lastlogin

Script Ends

Latest changes:

I added the 2 second wait times as Sxipper (a password plugin) seemed to be causing 'focus' problems.
ErrorIgnore is used so that the script still works (though slower) if you have logged off but are still on the Flickr site.

Leopard problems with OpenOffice

2008-10-22T11:56:00.004+01:00

Since installing an update from Apple to my X11 application, OpenOffice has refused to run. Damned annoying that Apple's development labs can't test updates properly.

Luckily I found a solution on a technical forum of running NeoOffice instead. NeoOffice uses exactly the same open source as Open Office, so saved files are completely interchangeable with OpenOffice, with the odd additional feature thrown in. NeoOffice runs natively on Leopard and so doesn't rely on X11. So far it seems to run rather better than Open Office and I have had great fun mapping out my kitchen using the drawing application (very similar features to Visio) which includes the facility to draw to scale, so my kitchen units can be represented quite accurately for my workmen to follow.

Next time I'm running updates from Apple I'll research the forums before rashly saying yes to new bugs.

Stumbling across some MacMini tips/tricks

2008-09-09T09:28:00.013+01:00

Tip 1 - Booting Puppy from CD
Having a good play around with Puppy Linux led me to getting this to work within Parallels running on Leopard with the MacMini (intel core duo). It succeeded quite well, neatly running Puppy directly from the ISO file, though it's a bit freaky being able to flick between 9 desktops on the Mac and a further 4 running on Puppy (note that my machine has been upgraded to 2GB of RAM so reserving 500MB for Puppy Linux was no problem). The only drawback was that Compiz Fusion would not run within Parallels (I really wanted to see the 3D cube working). I decided to try running from CD direct.

Booting from CD was not an obvious process as unlike a PC the Mac will not boot automatically from CD just because it's in the drive. The trick is to keep the Option key pressed when your machine powers up. It then checks bootable devices and gives you a neat graphic display of the options. Puppy then ran perfectly well including all the lovely 3D Compiz goodness. Though it's not a replacement for me working on Leopard it could be a really handy backup if I run into hard disk problems.

Footnotes:
I'm unsure if there's a way to get the boot up to work straight from a USB drive, this doesn't seem to be part of the standard boot up routine.
Parallels doesn't seem to let me share the Mac hard disk or USB drives with Puppy though I haven't spent any time researching a workaround.
The benefit of taking virtual machine snapshots is a key bonus, I can install options and change settings on puppy with the confidence that I can roll back to a happy working state.

Tip 2 - Ejecting a CD
While creating Puppy bootable CDs I had a problem with one not being read properly on the Mac (probably a scratched disk). Annoyingly it would not eject using software control. This included trying to get it to eject using direct commands from within Terminal. My final solution was to keep F12 pressed on the keyboard when powering up, magically the disk ejected. It seems an amazingly user unfriendly feature to have no manual eject process, not even a pin hole to work a paper-clip into, particularly on a machine which is so notoriously difficult to force open the case. From now on, I'm being extra careful to ensure disks look spotless before inserting them.

How to breathe life back into a 10 year old Compaq laptop using Xubuntu

2008-09-06T13:25:00.028+01:00

Many years ago I lent an old slightly damaged laptop to a friend (the laptop locking mechanism was broken and the screen slightly damaged). Last year he bought a cheap but highly functional laptop after my loaner, now nearly ten years old, stopped working and eventually returned it to me for possible repair and use.

It is a Compaq E500 which has a rather stylish black and grey layout, good quality keyboard and a clear colour screen (in 1999 it was retailing for $1,970 and in 2008 they sell on eBay for under $100). The hard disk had fatally crashed and windows now failed to boot up only ever giving a blue screen of death. I'm unsure if the hard disk has a physical problem but after it laying around on my study floor for six months I thought it might be possible to replace Windows with a suitable free small version of Linux so the machine could become a handy wireless web laptop and guest machine. [E500 Spec: 14.1 in 1024x786 TFT display (1600x1200 external), 12GB hard disk, CDROM 24x, Video Out, Pentium III 650MHz, 192MB SDRAM.]

After failing to get the standard Ubuntu to install from a CD I had to hand (I gave up after letting the disk whizz around for over a half hour) I did a little research and opted for Xubuntu which is a cut down version for machines with more limited resources. I downloaded and created a CD from the ISO file on my trusty MacMini. The E500 install took over an hour but the main screen gave a countdown bar so you can see something is happening. I re-booted when it told me and it worked first time. It boots up more quickly than it used to under Windows and had even recognized my 2GB USB flash drive!

After a little play to ensure that it seemed responsive enough I dug out an old Belkin wireless card (802.11b) and was quite astonished when I plugged it in and the OS used it to find wireless connections without any further configuration. I had to tweak the wifi settings to connect to my secure network but had it working as a wireless laptop fairly painlessly.1

It's now a smart "netbook" (with a much better screen than the Acer Aspire One) for checking email while watching TV. The remaining issues are hardware specific2 and I'm not sure I'll get around to fixing them, e.g. recognizing the function key buttons for changing brightness and volume & the second touch pad key doesn't do anything. All in all I'm pretty delighted to turn a useless brick into a nicely functional spare laptop.

Footnotes

[1] The old Belkin wireless card is so out of date that it can't support an encrypted connection so security on this laptop is reliant on screening by MAC address. In the process of testing it out I found wicd is a neat free Linux graphical tool for sniffing out and managing your wireless connection. I have just bought a cheap 802.11g card off eBay to solve the security problem.

[2] The hardware side remains a puzzle. The E500's BIOS is dated 1999 and there's a new BIOS that helps to solve the problem of buttons being available to non-Windows OS's (see discussion). Having reformatted the disk I can't install the new BIOS to flash (using a .exe file that HP have available) or (for some reason I can't fathom) run DOS from a floppy even with it set as the first boot device.

Postscript

I have tried booting from CD using the latest Puppy Linux (Dingo). It runs well, possibly faster than Xubuntu but I haven't managed to map the keyboard touchpad correctly yet. If I get this working I'll give it good trial as Xubuntu tends to take a couple of seconds to change screen tabs when running Opera, while I haven't noticed any real performance problems with Puppy Linux so far. When you think about it, it's pretty impressive to be able to run the latest version of the Opera browser on such an old machine.

How to add 8GB of memory to Sony Ericsson K750i

2008-08-09T09:40:00.014+01:00

My previous memory stick in my mobile phone was a 4GB MS Pro Duo card. This is the maximum the phone is supposed to be able to cope with. I took a slight risk and paid £23 for an 8GB MS2 card (with Pro Duo adapter) with the intent of extending my mobile's life as a thrifty guy's iPod.

Initially there seemed to be a problem reliably writing files but after re-formatting and using a card reader (rather than the phone cable) to update files, it seems to do the job. I formatted using DOS with the command FORMAT E: /FS:FAT32 /A:4096, check this discussion as to why I chose these options. The phone status only reports a 4GB card but it actually does cope with 8GB.

I'm amazed to fit such a large card on a three year old mobile phone. This is enough space to take a large MP3 music collection (about 50 CD's worth of music, including 15 hours of Wagner's Ring of the Nibelung), 40 podcast subscriptions, a couple of free audio books, various maps supporting my GPS bluetooth gadget, document backups and 2.5GB spare for future growth.

For document backups I use TrueCrypt to keep an encrypted 400MB disk image which can be mounted on my Mac or a PC that includes all my scanned documents over the last two years (I use a Fujitsu ScanSnap scanner to keep PDF versions of bank statements, tax bills etc.) as well as other key reference documents. This may seem an odd thing to keep on the phone but if I was on holiday and my laptop got stolen I could still print out identity documents if someone lent me a PC. By using encryption if the phone was stolen at least I wouldn't worry about my ID being compromised.

My old 4GB memory card? This is now in my pocket camera (Sony T100) giving enough space for 3 hours of 640x480 video or 1,600 photos at 2592x1944 (5MB) resolution.

How to write a spreadsheet formula for ISBN-10 to ISBN-13 and ASIN

2008-08-07T17:23:00.012+01:00

I wanted to convert 10 digit ISBNs to the new 13 digit format and vice versa for my "checking second-hand book prices given an ISBN" spreadsheet. It turns out that (for the moment) any ISBN-13 starting with 978 can be converted to an ISBN-10 and this will be the same as the ASIN used by Amazon, so this can be used to look up their data. The problem is that the last digit of the ISBN may change due to it being calculated using a different ISBN standard, so to convert from one to the other your spreadsheet needs to do the checksum calculation.

The following formulae are based on a 13 or 10 digit ISBN in cell A3 and cell A6 reserved for a check digit calculation. These should be in one line in a spreadsheet cell, the layout here is to help readability.

Formula to create ISBN-13 from ISBN-10

=if(
 len(A3)<11,
 concatenate(
   "978",
   mid(A3,1,9),
   mod(
     mid(A3,1,1)*3+
     mid(A3,2,1)+
     mid(A3,3,1)*3+
     mid(A3,4,1)+
     mid(A3,5,1)*3+
     mid(A3,6,1)+
     mid(A3,7,1)*3+
     mid(A3,8,1)+
     mid(A3,9,1)*3,
     10
     )
   ),
 A3
 )

Formula to create ISBN-10 from ISBN-13 (starting with 978)

=if(
len(A3)>10,
concatenate(
  mid(A3,4,9),
  if(A6=10,"X",A6)
  ),
A3
)

To calculate the check digit (A6)

=if(
len(A3)>10,
mod(
  mid(A3,4,1)+
  mid(A3,5,1)*2+
  mid(A3,6,1)*3+
  mid(A3,7,1)*4+
  mid(A3,8,1)*5+
  mid(A3,9,1)*6+
  mid(A3,10,1)*7+
  mid(A3,11,1)*8+
  mid(A3,12,1)*9,
  11
  ),
""
)

Track Flickr group membership history using iMacro

2008-05-27T12:39:00.001+01:00

User Guide	This script keeps a running log (every time you execute it), by updating a discussion message and/or saving a snapshot of group memberships. For the running log you have to create a message in each Flickr group you are an administrator for (or specifically declare), called "Member count". This log keeps a historical record of "date, membership total, total number of items". Before adding a new log entry it checks if the last entry (and the one before that) was less than a fortnight ago and updates it semi-intelligently.
Why?	I created this script as there is no easy alternative for keeping a history of membership for restricted groups in Flickr.
Notes	I'm an administrator for 9 Flickr groups and maintain membership logs in 4 other groups. Watching the macro chug along updating the logs is quite satisfying. It takes 10 seconds per group (loading 3 to 4 web pages each time and added waiting time, to avoid possible page freezes, composes most of that time) but this can increase up to three-fold depending on Flickr traffic. For a macro that I only need to run once every one or two weeks that doesn't seem too shabby.

[Source code now maintained at Sourceforge.net]

iMacro setup

iMacro installs itself with some example JavaScript files (.js) which are visible in the folders that iMacro shows you in Firefox.

If you find the location of these folders on your hard disk (on the Mac these are under /<your_username>/iMacros/Macros/) and then save the script in this location with a ".js" extension you should be able to see the file when you refresh the iMacro folder list in Firefox.

You should then be able to play the script from within iMacro (you have to already be logged in to your Flickr account and have created "Member count" discussion topics for groups you want to log). In the first instance I suggest you try the test test version which will run fully and show you the intended changes in the detailed change log but will not actually commit them.

See wiki.imacros.net for further help.

Flickr iMacro script to invite all your Flickr friends to a group

2008-05-26T10:19:00.010+01:00

There was no automatic way to invite all your friends to join a group so I knocked up this quick script to take you from any group main page to the invite page, extract your list of friends (and no other contacts) and then send off the invite. The invite only goes to friends not yet in the group and not previously invited so you will not be bugging them with unwanted spam.

Note that more than 500 friends may result in time-out errors waiting for Flickr to respond. In this situation you may be better off inviting a few hundred at a time by recording an iMacro selecting the first checkbox on the invite page, then generalize it by removing the ATTR=NAME: and copy that line 200 times but change POS=1 to POS=R1 so that the next checkbox relative to the cursor position is ticked each time. When you've trimmed to a more manageable number then this script can apply again.

[JavaScript source code START]

/*
Description:
JavaScript for iMacro to create a block invite for all your 
friends (but not family or other contacts) to join a Flickr group.
For more information see http://useroffline.blogspot.com

Assumptions:
- You have iMacro installed.
- You have your group's main page currently displayed.

Why:
There is no block invite facility built into Flickr.
*/

// Get friends list
var friendArr=new Array();
iimPlay("CODE:\
TAG POS=1 TYPE=A ATTR=TXT:Invite\n\
TAG POS=1 TYPE=A ATTR=TXT:Choose<SP>from<SP>your<SP>contacts?\n\
TAG POS=5 TYPE=TR EXTRACT=HTM\
");
// Tidy up extracted data
friendArr=iimGetLastExtract(1).split("name=\"");
friendArr.splice(0,1);
for(i in friendArr){
friendArr[i]=friendArr[i].split("\"")[0];
friendArr[i]="TAG POS=1 TYPE=INPUT:CHECKBOX ATTR=NAME:"+friendArr[i]+" CONTENT=YES";
}
iimPlay("CODE:"+
"' Total no. of friends = "+friendArr.length+"\n"+
friendArr.join("\n"));
// Send the message
iimPlay("CODE:\
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:Submit&&VALUE:PREVIEW*\n\
TAG POS=1 TYPE=INPUT:SUBMIT ATTR=NAME:done_send&&VALUE:SEND*\
");

[JavaScript source code END]

Re-list books on Amazon using iMacro

2008-05-22T22:14:00.002+01:00

User Guide	This script imports my Google spreadsheet of books and then pastes them to sell on Amazon books. It takes about 20 seconds to list each book but you only process those in your spreadsheet marked to be re-listed. Example Google Spreadsheet.
Why?	Unless you pay £28.75 per month to Amazon Books to become a professional seller, you don't get the facility to import from a spreadsheet. This iMacro code does exactly that, for free, though it is of course intended for smallish quantities (in practice I'm normally adding less than 6 books at a time and have less than 20 books up for sale). Having recently realized that the handful of books I was trying to sell had not only all expired but had also permanently gone from my closed listings so I had to put them in from scratch, I had the incentive to spend a few hours playing with JavaScript and iMacro to find a decent work around.
Known Bugs	When I have more than 20 books the last ones on the list are ignored as already listed, probably due to export limitations with iMacro. My work around is to list new books at the top of the spreadsheet and it will work fine. You could also cut and paste previously listed books into a second page of the spreadsheet and keep the front page for new books to be listed.


/*****************************************************************************
Title: AMAZON BOOK LIST IMPORT FROM GOOGLE SPREADSHEET

Created: 15 May 2008
Last Edit: 17 May 2008
Author:  UserOffline.blogspot.com
If you find this script useful, blog me back.

Description:
This iMacro script reads data from a Google spreadsheet of books and then
writes all those (marked as unlisted) to Amazon. Progress is highlighted
in a pop-up window.
The google spreadsheet has data for:
[0] ISBN,
[1] Title (for convenience only, not used),
[2] Condition,
[3] Comments,
[4] Price,
[5] Listed (Yes/No).
This script is particularly handy for managing books taking longer than
Amazon's 60 day listing expiry to sell.

Assumptions:
1. Google spreadsheet published and the URL GOTO command below points to it.
Note by publishing your spreadsheet, in theory anyone could read it; if
they can find it. As there is no personal data apart from the price you
set for the books (which is published on your Amazon listing) this creates
no privacy risk that I can think of apart from someone associating your
Amazon trading name with your Google account name. If you consider this a
risk I suggest you create a new (free) Google account for this purpose.
2. You are logged in to your Amazon account.
3. You don't use Firefox while it is running.
Note run time is about 25s/book but this includes pauses so you can see
what is going on, remove these to save another 5 to 10 seconds.
4. You have iMacro (it's free) installed with Firefox.
5. This script does not remove listings from Amazon, so if you want to change
a listing do it by hand or delete the old one and use this script to create
it again.

Version History:
17 May Changed ISBN recognition to recognize ASINs as well.
******************************************************************************/

//Define functions

function writeResults(s){top.results.document.write(s)}
function showTime(d){
var myS=parseInt((d.valueOf()-startDate.valueOf())/1000);
return parseInt(myS/60)+" minutes "+(myS % 60)+" seconds";
}
/* Pad number with zeros */
function pad(number, length){
var str=""+number;
while(str.length<length){str = "0"+str}
return str;
}
/* Make a string iMacro command compliant */
function safe(str){
return str.replace(/\s/g,"<SP>").replace(/\n/g,"<BR>");
}

//Open progress window

top.results=window.open( "", "myWindow", "menubar=1,resizable=1,status=1,\
width=300,height=450,scrollbars=1,left=10,top=500");
writeResults("<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \
\"http://www.w3.org/TR/html4/strict.dtd\">\
<html><head><title>Amazon book lister</title></head>\n\
<body bgcolor=lightyellow>\n\
<font face='Tahoma, Arial' size=1>\n\n");

//Variables

var cr="\n", i, j, startDate=new Date();
var books=new Array();  // 2 dim array for book data
var booksHTML=new Array(); // HTML rows from spreadsheet

//Get table of books and extract to booksHTML array

iimPlay("CODE:\
URL GOTO=http://spreadsheets.google.com/pub?\
*Your Google URL to a published spreadsheet goes here*\n\
TAG POS=3 TYPE=TABLE EXTRACT=HTML");
booksHTML=iimGetLastExtract(1).split("<tr");
j=0;
for(i in booksHTML){
/* check for ASIN/ISBN of 10+ digits or uppercase letters */
if(booksHTML[i].search(/[\dA-Z]{10}/)>-1){
books[j]=booksHTML[i].split("<td");
j++;
}
}
/* Note - I started off using </td to split the rows, which worked initially
but Google appears inconsistent in use of </td> as the next time I used it,
the </td> tags were sloppily not included in the web page source. */

//Tidy books array

for(i in books){
/* remove first chopped HTML tag (using "||" as a covenient marker) */
books[i]=books[i].join("||").replace(/\|[^\|<>]*>/g,"|").split("||");
/* drop first two elements as google spreadsheet chaff */
books[i].splice(0,2);
/* remove any leftover full HTML tag */
books[i]=books[i].join("||").replace(/<[^><\|]*>/g,"").split("||");
}

//Display and put details on Amazon

for(i in books){
if(books[i][5].search(/No/ig)>-1){
writeResults("<br>");
for(j in Array(0,1,2,3,4)){
writeResults(books[i][j]+"<br>\n");
}
iimPlay("CODE:\
URL GOTO=http://www.amazon.co.uk/gp/seller-account/management/your-account.html\n\
TAG POS=1 TYPE=A ATTR=ID:ListSingleItemsLink&&TXT:List<SP>single<SP>items\n\
'** ISBN **\n\
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:*asin CONTENT="+
books[i][0]+"\n\
WAIT SECONDS=1\n\
TAG POS=1 TYPE=INPUT:IMAGE ATTR=ID:*asin.submit\n\
'** Condition and comments **\n\
TAG POS=1 TYPE=SELECT ATTR=NAME:*condition-type CONTENT=$"+
safe(books[i][2])+"\n\
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:*condition-comments CONTENT="+
safe(books[i][3])+"\n\
WAIT SECONDS=1\n\
TAG POS=1 TYPE=INPUT:IMAGE\n\
'** Price **\n\
TAG POS=1 TYPE=INPUT:TEXT ATTR=NAME:selling-asking-price CONTENT="+
books[i][4].replace(/[^\d\.]/g,"")+"\n\
TAG POS=1 TYPE=INPUT:RADIO ATTR=VALUE:will-ship-to-united-states\n\
WAIT SECONDS=1\n\
TAG POS=1 TYPE=INPUT:IMAGE FORM=ACTION:/exec/varzea/sdp/sai-confirm*\n\
'** Final visual check **\n\
WAIT SECONDS=6\n\
TAG POS=2 TYPE=INPUT:IMAGE FORM=ACTION:/exec/varzea/sdp/sai-thank-you*\n\
");
writeResults("<span style='color:red;font-weight:bold'>[Book "+i+
" now listed on Amazon]</span><br>\n")
}else{
writeResults("<span style='color:green'>"+("[Book "+i+
" is already listed]</span>").replace(/\s/g,"&nbsp;")+" \n")
}
}

//Close down

var endDate=new Date();
writeResults('\n<p>Run time '+showTime(endDate)+"."+cr+
"<br>For more information see <a href='http://useroffline.blogspot.com'>\
UserOffline</a> blog.\n</font>\n</body>\n</html>");
top.results.document.close();
/* Could close when finished with top.results.window.close(); */

// END

How to add a book title from an ISBN or ASIN in a Google spreadsheet

2008-05-18T23:28:00.009+01:00

Having recently created an iMacro to import books from Google spreadsheet to Amazon, I realized that the formula I was re-using to automatically put in the book title was quite a neat feature of Google docs that most people are probably unaware of.

I use the importXML function. This finds data based on an HTML tag in any webpage and you can create that source webpage name as a formula. This gives a really neat way of cribbing data about a book straight from it's Amazon webpage to be used on your own spreadsheet (also known as website scraping).

Here's my example formula (assuming the ISBN or ASIN is at cell A1), read this as one line:

=importXML( concatenate( "http://www.amazon.co.uk/exec/obidos/ASIN/", A1), "//span[@id='btAsinTitle']")

By looking at the webpage source code for any Amazon book you can find HTML tags around all of the product data. In this case the HTML tag <span> with the ID attribute set to "btAsinTitle" surrounds the book title. If you wanted to grab data on the number of pages, publisher or even lowest second hand price, this is all possible in a similar manner.

I have also created neat formulas to grab best match full film title and release date from IMDB.com as a search sheet to support my DVD index so I can quickly cut & paste details based on typing in key words from the film title.

The only limitation is there is a maximum of 50 such formulas in any spreadsheet. However if you are building a catalogue or sales list, it is a fairly simple matter to cut & paste the values over the formulas for books or DVDs already indexed so only new entries use the formula.

How to find the cheapest train tickets by website scraping

2008-04-27T12:45:00.029+01:00

One of my niggles with train timetable websites is that it can be very tricky to find the cheapest train ticket if you want to ask questions like "What is the cheapest ticket if I could travel on Sunday, Monday or Tuesday next month?" or even worse "What would be the cheapest 1st class ticket if can travel any weekend in June?". If you phone up a train hotline they would probably give up (and you are normally paying 10p/min while they try) or they would guess and might miss the best deal.

My solution was to use iMacro. This is a macro recording plugin for Firefox that can automate interactions with a website. I can now set up an initial script and set the browser to look for the cheapest possible ticket.

For example, below are the results for London to Nottingham travelling in the mornings (9am to 1pm) from 3rd to 5th May searching for the cheapest direct train ticket in any 2 hour slot. It took 20 seconds to configure the script (after taking a day to develop it) and my computer took 5 minutes to run the search as it has to enter data on the timetable website 12 times (3 days x 2 time slots x two classes of travel) and wait each time for the website to refresh with new data. I chose the Virgin Trains website as it seemed to have quite a nice interface which made it easy to pick the first radio button to find the cheapest ticket and seemed to return the most comprehensive results (I kept finding discrepancies on other sites, which surprised me as I assumed they shared the same underlying data).

Results for London to Nottingham 
[click to show longer example]
Searched on Sun, 27 Apr 2008 12:44:50 GMT

Saturday  3 May 2008
Slot  | Time  |   Std   | Time  |   1st
09:00 | 08:55 | £ 15.00 | 08:55 | £ 18.00
11:00 | 12:55 | £ 15.00 | 11:55 | £ 18.00

Sunday  4 May 2008
Slot  | Time  |   Std   | Time  |   1st
09:00 | 09:00 | £ 15.00 | 09:00 | £ 18.00
11:00 | 11:00 | £ 15.00 | 11:00 | £ 18.00

Monday  5 May 2008
Slot  | Time  |   Std   | Time  |   1st
09:00 | 08:55 | £ 11.00 | 08:55 | £ 18.00
11:00 | 10:55 | £ 11.00 | 10:55 | £ 18.00

London to Nottingham
[click to show shorter example]

London to Nottingham

Virgin Trains Wed, 30 Apr 2008 23:26:26 GMT

Thursday 1 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 30.00	09:55	£ 30.00
10	09:55	£ 30.00	09:55	£ 30.00
12	11:55	£ 49.00	11:55	£ 86.00
14	13:55	£ 49.00	13:55	£ 86.00
16	15:55	£ 49.00	15:55	£ 86.00
18	19:30	£ 49.00	17:55	£ 86.00
Friday 2 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 22.00	09:55	£ 29.00
10	10:55	£ 18.00	10:55	£ 18.00
12	11:55	£ 18.00	11:55	£ 22.00
14	13:55	£ 22.00	13:55	£ 22.00
16	17:00	£ 45.00	17:00	£ 45.00
18	17:55	£ 45.00	17:55	£ 45.00
Saturday 3 May 2008
Slot	Time	Std	Time	1st
08	07:55	£ 18.00	07:55	£ 18.00
10	09:55	£ 22.00	09:55	£ 22.00
12	12:55	£ 15.00	12:55	£ 18.00
14	14:55	£ 15.00	13:55	£ 18.00
16	15:55	£ 15.00	15:55	£ 18.00
18	17:55	£ 18.00	17:55	£ 22.00
Sunday 4 May 2008
Slot	Time	Std	Time	1st
08	09:00	£ 18.00	09:00	£ 18.00
10	10:00	£ 18.00	10:00	£ 18.00
12	12:00	£ 18.00	12:00	£ 22.00
14	14:00	£ 22.00	14:00	£ 22.00
16	16:00	£ 22.00	16:00	£ 22.00
18	21:00	£ 18.00	20:00	£ 22.00
Monday 5 May 2008
Slot	Time	Std	Time	1st
08	07:55	£ 11.00	07:55	£ 18.00
10	09:55	£ 15.00	09:55	£ 22.00
12	12:55	£ 18.00	12:55	£ 22.00
14	13:55	£ 22.00	13:55	£ 29.00
16	15:55	£ 22.00	17:45	£ 22.00
18	17:55	£ 22.00	19:30	£ 22.00
Tuesday 6 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 15.00	09:55	£ 22.00
10	09:55	£ 15.00	09:55	£ 22.00
12	11:55	£ 15.00	11:55	£ 22.00
14	13:55	£ 18.00		£
16	15:55	£ 49.00	15:55	£ 86.00
18	19:55	£ 22.00	19:55	£ 29.00
Wednesday 7 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 15.00	09:55	£ 22.00
10	09:55	£ 15.00	09:55	£ 22.00
12	11:55	£ 15.00	11:55	£ 22.00
14	13:55	£ 18.00	13:55	£ 29.00
16	15:55	£ 49.00	15:55	£ 86.00
18	19:55	£ 18.00	19:55	£ 29.00
Thursday 8 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 15.00	09:55	£ 22.00
10	09:55	£ 15.00	09:55	£ 22.00
12	11:55	£ 15.00	11:55	£ 22.00
14	13:55	£ 18.00	13:55	£ 29.00
16	15:55	£ 49.00	15:55	£ 86.00
18	19:55	£ 22.00	19:55	£ 29.00
Friday 9 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 18.00	09:55	£ 22.00
10	11:55	£ 11.00	11:55	£ 18.00
12	11:55	£ 11.00	11:55	£ 18.00
14	13:55	£ 18.00	13:55	£ 22.00
16	17:00	£ 45.00	17:00	£ 45.00
18	19:55	£ 40.00	17:55	£ 45.00
Saturday 10 May 2008
Slot	Time	Std	Time	1st
08	07:55	£ 9.00	07:55	£ 14.00
10	11:55	£ 14.00	11:55	£ 14.00
12	12:55	£ 11.00	11:55	£ 14.00
14	13:55	£ 11.00	13:55	£ 14.00
16	15:55	£ 15.00	15:55	£ 18.00
18	17:55	£ 18.00	17:55	£ 18.00
Sunday 11 May 2008
Slot	Time	Std	Time	1st
08	09:00	£ 11.00	09:00	£ 14.00
10	10:00	£ 11.00	10:00	£ 14.00
12	12:00	£ 18.00	12:00	£ 22.00
14	14:00	£ 18.00	14:00	£ 22.00
16	16:00	£ 22.00	16:00	£ 22.00
18	21:00	£ 15.00	21:00	£ 18.00
Monday 12 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 9.00	09:55	£ 18.00
10	09:55	£ 9.00	09:55	£ 18.00
12	11:55	£ 49.00	11:55	£ 18.00
14	13:55	£ 15.00	13:55	£ 22.00
16	15:55	£ 40.00	15:55	£ 64.00
18	19:55	£ 18.00	19:55	£ 29.00
Tuesday 13 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 9.00	09:55	£ 18.00
10	09:55	£ 9.00	09:55	£ 18.00
12	11:55	£ 9.00	11:55	£ 18.00
14	13:55	£ 15.00	13:55	£ 22.00
16	15:55	£ 40.00	15:55	£ 64.00
18	19:55	£ 18.00	19:55	£ 29.00
Wednesday 14 May 2008
Slot	Time	Std	Time	1st
08	09:55	£ 9.00	09:55	£ 18.00
10	09:55	£ 9.00	09:55	£ 18.00
12	11:55	£ 9.00	11:55	£ 18.00
14	13:55	£ 15.00	13:55	£ 22.00
16	15:55	£ 40.00	15:55	£ 64.00
18	19:55	£ 18.00	19:55	£ 22.00

Search time 142 minutes 27 seconds

I have tested with very long queries taking more than an hour to run and the only problem may be time-outs from the website. In these cases iMacro appears to lock but by pressing the Pause/Resume button will continue without losing data. Note it is best not to use the browser when running an iMacro script but I have successfully used a different browser (Safari) at the same time without any problems.

The macro is run from a javascript file in iMacro which calls iMacro commands (previously I passed variables to an iim file but it seems easier to put it all in one file) and I tweak the "Set up query" section of the javascript to query a particular train route, this could easily be set from user prompts. For the time being it is always cheapest to get two singles rather than a return so I've only bothered writing the macro for a single. Note that the slot hours (set in array myhour) are based on the maximum number of hours that Virgin trains will display in the particular train route. For London/Nottingham this is 2 hours and for London/Redruth this is 3 hours. Nice bonus functionality I've included are returning the date in long text form from the Virgin site and keeping the iMacro code display updated telling you the estimated time left.

The latest version opens a results window to show the data in an html table. You can use iimDisplay() but it is limited to a tiny window and was (at the time of writing) not resizeable by scripting.

Here's the source code. Note that the source code is not word wrapped and long lines may appear truncated, but if you cut & paste to your editor you should see all the text.

JavaScript source code Virgin-Trains.js (click to show)

// Set up query

var myfirstday=10;
var mynodays=3;
var mymonth="05";
var myleaving="London";
var mygoing="Nottingham";
var myhour = new Array("09","11");
  /* Every 2 hours for Nottingham trains as they are every 30 mins */

/** No need to edit below this line if you only want to configure the query **
/************************************************************************************
  Author: http://useroffline.blogspot.com
  Created: 28 April 2008
  Last Edit: 1 May 2008
  Title: iMacro Virgin Trains timetable search

  Description:
  Fills out query form on Virgin Trains timetable website (covers all UK timetables)
  and extracts the cheapest standard and first class ticket prices and times in any
  timeslot (normally two or three hour slots depending on what the timetables look
  like from Virgin for the particular route).
  Results are displayed in a new browser window in an html table which is written to
  as the search completes each calendar day.

  Assumptions:
  1. Virgin trains site has no technical difficulties
  2. iMacro has 'view Javascript while running' option set to off (for better speed)
  3. Infrequent usage to avoid IP blocking by Virgin
  4. Browser will not be used for anything else while running the macro
  5. Route does not use the new timetable layout (Virgin is upgrading layout)

  Suggestions for improvement:
  o Add link to each train in results table to live timetable page (probably not
    Virgin)
************************************************************************************/

// Open results window

var cr = "\r\n";
top.results=window.open( "", "myWindow", "menubar=1,resizable=1,status=1,width=500,height=600,scrollbars=1");
top.results.document.write("<html><title>Train Timetable search results</title><head>"+cr+
  "<style>td {border-width:thin;border-color:black;-moz-border-radius: 8px 8px 8px 8px;}"+cr+
  "</style></head>"+cr+
  "<body bgcolor=lightgreen style='font-family:Tahoma, Arial;'>");
top.results.window.status="Initializing, time to make a cup of tea.";

// Variables

var c=0, j, price=" na ", count=0, mytime=" na ", longdate=" na ", pArr=new Array();
var classArr=new Array("Std","1st");
var startDate=new Date();
var ret=cr+"<h3>"+myleaving+" to "+mygoing+"</h3>"+cr+"<br>Virgin Trains "+startDate.toGMTString()+"<p>"+cr;
var retTemp, runTimeM, runTimeS, runTimeMS, timeLeftS, timeLeftM, loopTime;
var mydate = new Array;

// Pad dates ("1" becomes "01")

var i;
for (i=0;i < mynodays;i++) {
  mydate[i]=(myfirstday+i).toString();
  if (mydate[i].length < 2) { mydate[i]='0'+mydate[i] }
  }
var loopTotal=mydate.length*myhour.length*2;

// Initialize to avoid timeout errors

iimPlay("CODE:URL GOTO=http://www.virgintrains.co.uk");

// Main loop

ret+="<table class='results' bgcolor=lightyellow cellpadding=4>";
top.results.document.write(ret);
ret="";

/* Static parts of the code */
pArr[1]="URL GOTO=http://www.virgintrains.co.uk/default.aspx";
pArr[7]="TAG POS=1 TYPE=SELECT FORM=NAME:qtt ATTR=NAME:outMinuteField CONTENT=$00"+cr+
    "TAG POS=1 TYPE=INPUT:IMAGE FORM=NAME:qtt ATTR=ID:_ctl15_findTimes";
pArr[9]="TAG POS=1 TYPE=INPUT:RADIO FORM=NAME:Form ATTR=ID:*ServiceTypeRadioButtons_1"+cr+
    /* ServiceType 1 = direct trains only, 0 = Any train */
    "TAG POS=1 TYPE=INPUT:IMAGE FORM=NAME:Form ATTR=ALT:Check<SP>availability"+cr+
    "TAG POS=1 TYPE=INPUT:RADIO EXTRACT=TXT"+cr+
    "TAG POS=1 TYPE=TD ATTR=CLASS:tableCell EXTRACT=TXT"+cr+
    "TAG POS=2 TYPE=TD ATTR=CLASS:tableCell EXTRACT=TXT"+cr+
    "TAG POS=3 TYPE=TD ATTR=CLASS:tableCell EXTRACT=TXT"+cr+
    "TAG POS=4 TYPE=TD ATTR=CLASS:tableCell EXTRACT=TXT"+cr+
    "TAG POS=5 TYPE=TD ATTR=CLASS:tableCell EXTRACT=TXT"+cr+
    "TAG POS=1 TYPE=TD ATTR=CLASS:tableTitle EXTRACT=HTM"+cr+
    "WAIT SECONDS=4";  // Could help to avoid lock-up?

for (j in mydate) {
  retTemp='<tr align=center ><td bgcolor=lightgreen >Slot</td><td bgcolor=lightgreen >Time</td><td bgcolor=lightgreen >Std</td><td bgcolor=lightgreen >Time</td><td bgcolor=lightgreen >1st</td></tr>';
  for (i in myhour) {
  retTemp+='<tr align=center ><td bgcolor=lightgreen >'+myhour[i]+'</td>';
  for (c=0;c < 2;c++) {
    count++;
    /* Estimate time left */
    var now = new Date();
    runTimeMS=now.valueOf()-startDate.valueOf();
    runTimeS=parseInt(runTimeMS/1000);
    runTimeM=parseInt(runTimeS/60);
    runTimeS %=60;
    loopTime=runTimeMS/count;
    timeLeftS=parseInt((loopTotal+1-count)*loopTime/1000);
    timeLeftM=parseInt(timeLeftS/60);
    timeLeftS %=60;
    /* Results window status bar feedback */
    top.results.window.status = "Loop "+count+" of "+loopTotal+
    " [Last result "+longdate+" @ "+mytime+", "+classArr[1-c]+" = £"+price.replace(/ /g,"")+"]";
    /* Run iMacro */
    pArr[0]="CODE:"+"' "+runTimeM+" minutes "+runTimeS+" seconds current run time"+cr+
        "' "+timeLeftM+" minutes "+timeLeftS+" seconds estimate to complete";
    pArr[2]="TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:qtt ATTR=ID:idLeaving CONTENT="+myleaving;
    pArr[3]="TAG POS=1 TYPE=INPUT:TEXT FORM=NAME:qtt ATTR=ID:idGoing CONTENT="+mygoing;
    pArr[4]="TAG POS=1 TYPE=SELECT FORM=NAME:qtt ATTR=NAME:outDay CONTENT=$"+mydate[j];
    pArr[5]="TAG POS=1 TYPE=SELECT FORM=NAME:qtt ATTR=NAME:outMth CONTENT=$"+mymonth;
    pArr[6]="TAG POS=1 TYPE=SELECT FORM=NAME:qtt ATTR=NAME:outHourField CONTENT=$"+myhour[i];
    pArr[8]="TAG POS=1 TYPE=INPUT:RADIO FORM=NAME:Form ATTR=ID:*TicketClassRadioButtons_"+c.toString();
    iimPlay(pArr.join(cr));
    /* Get long date */
    try {longdate=iimGetLastExtract(7).split('>')[2].replace(/  /g," ")}
    catch (err) {}
    /* Parse train time and price */
    price=iimGetLastExtract(1).substr(5+(iimGetLastExtract(1).substr(4)).search("_"));
    price='    '.substr(1,6-price.length)+price;      // Pad out to 6 characters
    mytime=parseInt(iimGetLastExtract(1).substr(0,1));  // Find the table column number of train time
    mytime=iimGetLastExtract(mytime+1);          // Get time in column header
    retTemp+='<td>'+mytime+'</td><td>£'+price+'</td>';
  }  /* end class */
  retTemp+='</tr>';
  }  /* end hour */
  ret+="<tr><td colspan=5 bgcolor=lightgreen >"+longdate+'</td></tr>'+retTemp;
  top.results.document.write(ret);
  ret='';
}  /* end date */

// Calculate total running time

var endDate=new Date();
var runTimeS=parseInt((endDate.valueOf()-startDate.valueOf())/1000);
var runTimeM=parseInt(runTimeS/60);
runTimeS %=60;
ret+='</table>'+cr+'<p>Search time '+runTimeM+' minutes '+runTimeS+' seconds';
ret+=cr+"<br>Created by <a href='http://useroffline.blogspot.com' title='Author details'>UserOffline</a> 2008";

// Display Results

top.results.window.status="Finished";
top.results.document.write(ret+cr+"</body></html>");
top.results.document.close();

How to use free GPS on a K750i mobile phone

2008-04-14T09:11:00.017+01:00

Google high level map

Last year I bought a tiny bluetooth GPS device off ebay for £20. I set up my laptop with it and we managed to drive around Germany using a copy of Autoroute showing our planned route, current GPS coordinates and it even shouted helpful instructions such as "turn left in 400 meters"! We planned our route in the hotel the night before and as we are often looking for poorly road-signed ancient sites it is terribly handy to have the confidence of the computer pointing you down an empty track rather than guesswork.

Open Street Map

I did get the GPS recognized by my Sony Ericsson K750i (due to a previous hack it runs using the W800i firmware) and used some trial software to show coordinates but little else. I spent some time yesterday further researching and managed to get it working quite beautifully yesterday.

The simple free java application is called TrekBuddy and it has an associated wiki site. After connecting to the GPS via bluetooth, the application shows where you are on a scrolling map that you define and store locally on the memory stick. I have it working with an almost A to Z street level map of London (spanning West Hammersmith through to Catford). You can load other maps as you go along so a later improvement would be to install a higher level road map of the UK. In the case of my map of London it was only 2mb in size (I use a 4gb stick!) so it would be no issue to carry around several maps.

Google low level map

I had to spend some time tweaking the configuration, in particular the initial location of the maps had to be entered by hand to point to the memory stick ("file:///E:/other/mapdata/london") and it took a while to work out that I needed to tick the option to enable a large atlas in order to display anything. The only other hard part was preparing the map, luckily someone has worked all this out and I imported maps from Open Street Map (setting default datum as WGS 84) using the tool listed on the wiki. I also managed to import using the rather neat Google Maps to TrekBuddy tool on the same page. The latter being slightly more limited as to the overall size of map but with the benefits of a better quality street map (just right for a pedestrian map in Central London).

Talktalk (Carphone) compromised my password!

2008-02-14T23:52:00.008Z

I had been getting mysterious emails to someone else that I thought was spam nonsense. This morning, after getting a few about membership of the same club, I emailed the club secretary back suggesting they had the wrong email (assuming some sort of typo).

This afternoon Talktalk Technical Support phoned me to apologise for giving out my password! Over Christmas my email had stopped working and I called to check why and they "reset" my account somehow and I agreed a new default password. I tried logging in afterwards to the online account manager but had lost the password they send you in the post to set up your account and after hanging on the 0870 number for a couple of minutes I gave up and left the password as their default random looking one. It turns out that a new customer in January had been given my email account details and the password as their account. Consequently that customer has been logging into my email account over the last few weeks, probably thinking that email to me was spam and sending email from my address!

I can't believe that a company can be so cavalier about customer privacy. Needless to say, I've gone online and changed the password again from the new one agreed on the phone.

Lessons learned:

Never send anything confidential by email unencrypted (at least put a password on a document)
Always change passwords from defaults or if given temporarily to someone else
Stick to more secure mixed numbers and letters passwords of a decent length (8 characters or more), a tool like Sxipper can be helpful if you have to keep switching between accounts
If it's important then digitally sign it
Expect companies (like Carphone) to give away your details by accident and know who to contact if you think something has been compromised

Mac mini - six things they did not tell you about switching from PC to Macintosh

2008-02-12T18:01:00.000Z

Last November, after my PC died for about the 4th time I knew there was something seriously wrong, probably something odd happening with the motherboard. I thought I'd treat myself to something different so I splashed out on a Mac mini (Core Duo).

Tip: If you are buying a Mac from Apple, consider the cost of memory upgrades. I would have opted for a 2GB Mac mini rather than the standard 1GB version but they wanted £80 for the privilege. The actual memory cost from Crucial.com is £32 and when I sold my old memory on eBay for £12 I realized I had saved myself £60!

Mostly it's worked as promised but the following painful annoyances rather caught me out:

Parallels did not come as standard, something I really needed for running some of my old windows applications. My only other option would have been to reserve a major chunk of my hard disk and use dual booting, something that I've had real issues with in the past. I took a trial copy of Parallels and it works fairly well apart from being too slow. In fact so slow that in January I upgraded the memory (to 2GB), though as I'm techie enough to do this myself it was under 30 quid.

Webcam did not work. This has been a headache. Not only did my Philips USB webcam not work, there are no valid drivers to make it work (I wasted hours trying to get a generic driver to work). Even worse, there are no guarantees that any other cheap camera I can find will actually work. I still don't have a replacement webcam.

Scanner did not work. I have a rather expensive Fujitsu ScanSnap scanner. Unfortunately there are no drivers that will make it work natively in Leopard. I eventually got it running under Windows 2000 in Parallels, hardly ideal.

EVE Online does not work. I tried out a free account with EVE Online (a well known and popular multi-player game for the Mac), sadly it doesn't run reliably on the Mac mini and freezes up randomly.

Can't network with a PC. This is particularly annoying. I've been trying for several weeks to get this working. I have worked through inbuilt help and googled for solutions to no avail. I can get my Mac to recognize my home networked PC and even read/write files to its hard disk but not the reverse. I wanted this to work as I'm using a 500GB USB drive as my Time Machine and I've kept space on it for PC backups but as the Mac remains invisible to the PC my archive/backup process is frustratingly manual and I can't share a central file library.

Can't use my microphone/headphone. This was not made clear in the summary specification and is a really stupid design flaw. The audio-in socket is actually only for digital microphones. This means that my headphone microphones will not work (though confusingly they fit the socket and it took me quite a while to find out this was not some sort of driver problem). Consequently since buying the Mac I have not been able to use Skype, a real problem as I use this as my home office number. I have been searching for a second-hand iMic device (they're a bit pricey new), a USB adaptor for microphones, I should be able to find one for under £15. The alternative would have been to get a bluetooth microphone, but I can't bear having yet another device to keep charged up.