me, my vps and the beeb

Sometime ago, I decided to get a VPS as a web hosting solution. upon discovering an interest for formula one, I then begin on an epic quest to discover methods of watching it without getting a cable subscription. as it turns out, using a VPS as a proxy works as a solution for most UK media. namely the BBC’s iPlayer. however, there was one primary problem, i could not access live sports feeds and live video feeds.

My initial methods of getting a proxy going revolved around ssh-ing to the VPS as demonstrated here. However, to my chagrin, I could not watch live F1 – while the audio and radio broadcasts remained fine, iPlayer refused to load, with the ambiguous error message “The content could not be loaded. Please try again later.”. Try again later I did, and fail again later I experienced.

This sent me on a quest to attempt to discover what may be the underlying cause of this oddity. It clearly cannot be a result of the browser settings, given that to other sites and the beeb itself, my browser plays media fine (apart from live content). So this issue must be unique to the live media – which implies that there must be another method of discovering that my browser is using a proxy.

Some googling brought further light on the situation. It turned out that people trying to watch Hulu in the US experienced the same issues, and the main thing in common is that both of them use Flash player. It turns out that one of the idiosyncrasy of Flash is that the RTMP protocol uses stateful TCP connections and the resulting “bug” is that RTMP videos may not be available to computers behind proxy servers. In this case, this is used as a clever method of detecting if a client is using a proxy server as since RTMP uses direct TCP, it bypasses the proxy server settings of the browser and thus attempts to connect to the media directly through the current connection as stated here.

Some proposed solutions involve using foxyproxy, but I’ve decided to use Proxifier. Proxifier can be used on an 31 day evaluation basis. More details can be found here and here (it’s for mac but it works on a similar principle). Upon getting Proxifier, I quickly had a go at trying to configure my proxy – and it worked! I got BBC1 going on live broadcast, and if nothing really changes, I should theoretically get to watch the Turkish GP on the Beeb this weekend! Win!

Advertisement

the table fan

a decision i made 5 years ago has returned to bite me in the arse last night. whilst playing l4d2 with a couple of friends my graphics card decided that it was a little too hot, resulting in my PC giving up the ghost. of course, the reason behind this is simple – i forgot to turn my table fan on.

Yes, table fan. 5 years ago, when assembling the computer, I decided to passively cool my graphics card with a coolermaster coolviva heatsink. This seemed to be a good idea at the time since passive, silent computing was the “in” thing in 2005. Fast forward to 2010, when I’m playing L4D with friends. Usually I have a table fan pointed towards my open case in order to provide some degree of auxiliary cooling. On that faithful night, I forgot to switch on said fan. What happened afterwards was quick, simple and painful. Also expensive – the new card cost me $109.24.

The dead card was a XFX 6600GT. It was replaced with a EN220 from ASUS. One of the things that I’m supremely thankful for in this case was the purchase of a motherboard with PCI-E slots – the first of its kind in 2005.

I guess there’s a fringe benefit to the death of my graphics card. I no longer need to have the table fan on whilst gaming.

On LEDs

I was trying to sleep one night when this sprung to my mind, or rather my eyes. What is it with accessory manufacturers making each and every status LED on their product as bright as possible!

I bought a Vantec Nexstar enclosure some time ago for my extra hard drives and when I turned it on I was greeted by a blue light so bright Jupiter could spot it. This was tolerable as long as it was in the day, however at night, there was this constant blue aura which kept me up! (I can’t sleep when its not completely pitch dark, unless I’m in a car). This seems to be fairly consistent with chinese design/made gadgets.

Dear Manufacturers, you’re building a HDD enclosure, casing or network switch not a torchlight! For goodness sake, do not spec your LEDs at their theoratical maximum, but rather leave them at a level where they’re not intrusive to the workspace environment! Apple is a company that has pulled this off well, so has Nokia.

On that note, I finally threw in the straw and put a 680 ohm resistor in line with the LED on the enclosure, effectively dimming it to a more acceptable level.

Electronics in New Zealand

So you’re building your next electronic/electrical widget/FSAE car, and need to get components while you’re in NZ. Here’s a list of where you can potentially buy stuff from. This list has been compiled through many years of FSAE experience and may omit certain suppliers, but what’s crucial is here.

Components

RS New Zealand

The principal supplier of the team, RS is the first stop for the team’s electrical requirements. What can you get from RS? Heatshrink kits, resistors, crystals, regulators and all the components required to populate your PCB. The benefit of RS is that its based locally and can provide additional stock from US and UK should there not be enough in Sydney and NZ. Another benefit of RS is that they operate a trade counter in Mount Wellington which is easy to get to, and any items in stock can be quickly obtained from the trade counter. Also remember to check if they have any freebie promotions going on, RS often gives free toys for purchases over a set amount. Shipping is $10 for less than $125, and free for orders above $125. RS also supplies most of our engine sensors.

Mouser/Digikey/Farnell

As far as I know, these companies do not have a physical local presence within Auckland, however they provide free shipping for orders above $125, and $200 for digikey. We only buy from them when RS does not have the part in question. Prices are usually close enough for most components.

Connectors

connector technologies

Connector technologies is where we go to for our heavy duty connectors such as DTMs, Mil Spec connectors. They are based in Wellington and as such you have to account for the lead time when buying from them. Most items are ex-stock and we get our DTM connectors, DTM pins from them. Prompt response and service are available and they do stock other brands of connectors which we have yet to investigate or use.

connector systems

Connector Systems is the official distributor for Lemo connectors in New Zealand. They can order any Lemo connector that you can find on the website, but the issue with specialised connectors is that they have a long lead time. One of the connectors that we ordered last year had a lead time of about 3-4 months! Company is reasonably reliable in terms of shipping.

widgets/breakout boards/tools

adafruit

a store started and launched by a hardware hacker, adafruit is where we got our programmers from. The USBTiny programmer that we purchased from adafruit is pretty much the most fully featured programmer for its price on the market. Not to mention that the hardware and software is pretty much open source! Adafruit also sells Arduinos which might be useful for prototyping in the future, and the adafruit blog provides many updates on the current “electronics hacker” environment.

sparkfun/mindkits.co.nz

Another place where we get our widgets from, Sparkfun stocks many useful things such as breakout boards for accelerometers, gyros and microcontrollers. They also stock programmers and useful components such as switches, touch screens and LCDs. They also have a good amount of tutorials available on their website for free. I personally found the tutorials to be useful (3.3v interfacing)  and they should be considered a good place to look at if you have a question.

Mindkits.co.nz is Sparkfun’s New Zealand distributor. They sell Sparkfun products at a small margin, and often have group buys in which people can slot in their orders and get reduced shipping. This should be your first stop if you need a Sparkfun product and are located in New Zealand. They are also the host of the New Zealand Auckland Arduino User Group. 

heavy duty/generic

JA Russell

JA Russell is located near the workshop on Morrin Road. They sell mostly electricians equipment and supplies and not small electronics. They are a good place to get cable lugs from, and have a good stock of heatshrink too. If you ask nicely, they might let you use their crimping tool, which is useful since it allows you to get a decent connection on your wires and cable lugs.

Hella NZ

Hella NZ are the distributors for Hella in NZ, they also do some manufacturing within New Zealand itself. Their primary business lie in safety switches, trailer lights, trailer connectors and accessories and general automotive accessories. We got our brake light (actually a trailer running lamp), ISO Switch (Giant switch for all circuits within the car), cable tie resources and cigarette lighter sockets from Hella NZ.

Ideal Electronics

Ideal Electronics is located down on Lunn Ave, and we usually buy adhesive heatshrink and cable lugs from them.

Surplustronics/Jaycar/Dick Smith

Located on Upper Queen Street, Surplustronics has all sorts of generic widgets (wireless transmitters), generic components (if you don’t need 10 of them – RS has a MOQ) and tools like soldering irons, wire strippers etc. 

Dick Smith does not stock very many electronic goods these days, but they are a potential stop should you need something really urgently. Jaycar is primarily good for wire, loose components.

specialised components

will update this

SEEED

Just sent off a PCB to seeedstudio.com for production. Their pricing does seem to be relatively reasonable, at $50USD for 10 PCBs including shipping. Getting the PCBs submitted was just a matter of sending the gerber files off to them, and one of their engineers contacted me within 24 hours! on a saturday to boot.

I’ve just paid $60 (+$10 for white PCBs) for 10 PCBs, and will be waiting on them. Will put pics up as they come!

Update! They just shipped the PCBs

7 Nov submitted order
11 Nov Shipped.

Useful Trick #1

How to reset your AVR without having to powercycle – just run the basic avrdude tool

i.e. avrdude -c usbtiny -p c128

In the process of finding the device and retrieving the signature byte, it also resets the micro.

Handy!

semi stressed PCBs

While working at a now defunct company in 2007, I had a chat with an engineer with regards to PCB design and hobbyist robot construction. Back then , I was rather new to the industry, being a second year undergraduate. At the end of the internship, I took away many experiences and thoughts, but the most interesting lesson learnt was that of semi-stressed PCBs. This may not be new to industry, but I found it to be especially suited to that of the hobbyist industry.

The propective hobbyist can often be characterised by the following requirements – he has little to no access to heavy machinery, fabrication equipment. However, he is willing to outsource certain aspects of his designs, in this case, PCBs. The batchPCB service, or seeed’s propaganda service offer numerous rapid prototyping facilities for printed circuit boards.

The engineer in question was designing a robot for use by secondary school students, and one of the selling points was that all structural parts of the chassis, excluding nuts and bolts, were made from PCBs. His rationale behind it was that he could get the “boards” made with 0.2mm precision, holes predrilled and at a cost comparable to getting a sheet of metal and waterjet cutting the parts out. Fair enough! He used copper fills on both the top and bottom layers for stiffness in the boards. This may not be directly applicable to the traditional hobbyist, but there is potential for the hobbyist in this field. For example, the motherboard for your robot/device may be structural to the robot, being the top subframe. Stiffness and reinforcement in required directions can be achieved by means of copper fills.

However, a risk behind using this method is that solder joints may be susceptible to fatigue cracks due to undue flex in the board, it is essential to consider this in the design process. One solution would be to have some level of damping, on the board, or maybe another subboard as a support. The hobbyist, building a PCB for use in his own home may not have to bother about this though, considering that limited run boards aren’t required to have rock solid reliability with an MTBF of 42582350327503 hours.

I’m sure that this has already been implemented in industry practice, but applications in the hobbyist ecosystem have seen little use of this method (or as much as I’ve seen anyway). The result of using such a method was a professional looking robot that was basically snap together, with near perfect tolerances.

The robot in question looks something like this, but I will continue hunting for better examples.

Rapid downloading

An activity I’ve been engaging in now that its the end of semester is grabbing the notes for courses that look interesting to me and using them to decide if I want to take them next semester. The computer science department often posts the notes for courses on their website (publicly viewable!) across the entire semester, and at the end a nice collection of notes can be downloaded.

Therein lies the biggest problem, having to try to save and download 12 weeks X 3 lectures a week of notes is a real pain especially if I have to grab 5-6 different courses. So I set about writing a python script to help me out. Unfortunately, I only discovered python’s “websucker.py” script after I was midway through that bit of code. Meh.

First, I had to decompose the html script and find all the links present on a set page, then identify which of these links are .pdf/.ppt files (what I want to download), then download and save them to a local directory. Sounds simple, but in the end it took me about 3 hours in all to finish the script and get it working as I wanted it to. Currently, its in a working state, but nothing too pretty.

I used  BeautifulSoup as a html parser, a somewhat odd name but I’ve seen worse – e.g. CherryPy. I chanced upon it while googling around for the best option of parsing and searching through HTML trees and found a snippet of useful code on StackOverflow (the next best thing after experts exchange).

I started off with importing the required libraries

import urlparse, urllib
from BeautifulSoup import BeautifulSoup
import re

then opened the website, parsed it in

url  = "http://www.cs.auckland.ac.nz/compsci340s2c/exams/"
source = urllib.urlopen(url).read()

soup = BeautifulSoup(source)
print soup.prettify()

BeautifulSoup(source) takes the website and parses it into discreet tags that can be referenced. Also, the soup object has many different operations that can be carried out on it as seen later. soup.prettify is used to print the html code of the page, indented properly (hence the prettify moniker)!

link = 'empty'
links = []
filenames = [] 
for item in soup.findAll('a'):
    if (str(item).find('pdf') > 0):
        print item.renderContents()
        link =  str(item['href'])
        links.append(link)
        filenames.append(link)
        print 'pdf found'

This bit of code involves searching for all links and appending them to a list. I should have used a dictionary, but thats for version 2.0. soup.findAll(‘a’) looks for all tags with the label ‘a’. I then get all objects with the extension “pdf” in them and from there, I obtain the value of the attribute ‘href’ with item[‘href’], thank you BeautifulSoup!  The renderContents function shows the actual information stored within the tags.

 

Finally, we download all the files,

for index, item in enumerate(links):
    item  =  url + item
    print item
    print filenames[index]
    filename = "c:\\cs340\\" + filenames[index]
    print filename
    urllib.urlretrieve(item, filename)
    #print 'link is'
print 'finito'

Currently, the directory filepath is hardcoded, but that should really be softcoded (version 2.0 again!), the filenames are identical to that of the original files hosted on the webpage.

Downloading scripts after, we get the following output

Click to access 2007examWithAnswers.pdf

2007examWithAnswers.pdf
c:\cs340\2007examWithAnswers.pdf

Click to access 2008examWithAnswers.pdf

2008examWithAnswers.pdf
c:\cs340\2008examWithAnswers.pdf
finito


and we’re done! finito! Now, to put this to a UI, and build version 2.0. I’ve had a real crap time trying to post code here and preserving indenting, but using the “pre” and “code” tags seem to work