Maximum likely

Wine region analysis

2011-09-26T11:02:00.000+01:00

I have done some analysis on wine regions I am interested in, using wine-searcher.com data. I used all vintages of the top 500 wines (sometimes less) wine on their site. Apart from price, I also recorded Decanter awards. My analysis shows that Argentina has the best returns per rating point (ratings are an average of wine critics and the public, I have used points above 70). Piemont and Mosel have the worst stats: expensive or few awards. South Australia dominated among awards.

Region	Wines	Avg price	Median price	Awards	Commended	Net	Avg point per pound	Awards per wine
Alsace	2964	30	18	60	3	57	0.935	0.019
Argentina	2458	15	10	273	78	195	1.597	0.079
Greece	733	14	11	104	17	87	1.460	0.119
Mosel	3099	58	18	36	8	28	1.066	0.009
Piemont	5684	66	39	123	28	95	0.595	0.017
S Australia	4475	43	23	724	172	552	0.948	0.123
Stellenbosch	2425	17	11	353	100	253	1.460	0.104
Tuscany	5547	52	30	324	92	232	0.772	0.042
	27385			1997	498	1499

Classical music documentaries

2011-08-25T09:20:00.000+01:00

Bar/line chart improvement

2011-06-17T15:22:00.000+01:00

Bar/line charts are quite useful when we want to show the development of 2 variables on different scales or units of measurement. By default Excel gives you the left chart, which is kind of ok, but I have 'developped' a better version on the right, which limits the charts at the min and max of the each series and draws a line in the mid point of the range (Median or average won't work here because they might not be on one line).

Also note how the colour font on the axis labels gets rid of any legend box.

How to find R help online

2011-04-17T08:27:00.000+01:00

In R you can find the help page of a function by typing help(func). If you want to something quickly online, place library and function in the following URL and off you go.

http://127.0.0.1:21798/library/[lib]/html/[func].html

Does Amazon filter Kindle items well?

2011-04-15T16:16:00.000+01:00

As you might suspect, my answer is No. When searching for new Kindle books, I hardly find good results or recommendations. The problem is that there are a lot of virtually zero priced items which are top of the list but they are hardly worth the megabytes they carry. I am tired of looking at lists of cheap self help books.

Amazon seems to use the same recommendation idea for Kindle books but actually needs to adjust it to make it relevant. How does it help me if every second recommendation is Dracula just because it's free?

Amazon needs to put quality at the top of the list.

Better Choices Better Deals

2011-04-15T08:16:00.000+01:00

BIS has published a paper which outlines how customers can benefit from using their data to optimise their shopping. They quote the many loyalty cards and tools which are already out there. They are creating the mydata initiative where customers can access their own data and find the best deals based on their usage.

I am quote skeptic about this scheme.

Data is collected for a reason by specialised companies which exploit the data (not the customer), it is their asset.
Data formats from different providers/retailers are vastly different and will never be brought under one roof, if it will the data will lose its richness. (A good example is the number of households quoted from Boots, Tesco and Nectar - they all use different definitions of what an active household is)
There is a cost involved and it is not clear who will carry that.

Nevertheless I like the idea that customers have more rights to accessing their own data.

http://www.bis.gov.uk/assets/biscore/consumer-issues/docs/b/11-749-better-choices-better-deals-consumers-powering-growth.pdf

R and Python

2011-04-06T22:39:00.000+01:00

Here is a R and Python syntax table, I have also included Numpy commands to make it more comparable. Where a cell is empty I could not find an equivalent.

Task	Python	Python Numpy	R
sequence	x=[I for I in range(1,11)]		x <- 1:10
scalar	x=1	x=array(1)	x <- 1
vector/list	x=[1,2]	x=array((1,2))	x <- c(1,2)
constant vector	x=100*[1]	x=ones(100)	x <- rep(1,100)
append	x.append(1)		x <- c(x,1)
matrix	x=[[1,2],[3,4]]	mat([[1,2],[3,4]])	x <- matrix(c(1,2,3,4),ncol=2,byrow=TRUE))
column stack		hstack((x,y))	cbind(x,y)
row stack		vstack((x,y))	rbind(x,y)
for	for I in range(1,11): print I		for I in c(1:10)) { print(I) }
while	I=1 while (I<10): I+=1		I <- 1 while (I<10) { I <- I+1 }
if	if I==10: print 'Yes' else: print 'No'		if (I==10) { print('Yes') } else { print ('No') }
length	len(x)	len(x)	length(x), nrow(x)
columns	len(x[0])	x.shape[1]	ncol(x)
dimension		x.shape	dim(x)
summary			summary(x)
read csv	import csv reader=csv.read(open('file','r')) mydata=[] for line in reader: mydata.append(line)		mydata <- read.csv("file", header=TRUE)
write csv	import csv writer=csv.writer(open('file','w')) for d in data: writer.writerow(d)		write.csv(data, file="file", row.names = FALSE)
sum	sum(x)	sum(x)	sum(x)
select element	x[1][1]	x[1,1]	x[2,2]
last element	x[-1]	x[-1]	x[-1]
select column		x[:,1]	x[,2], x$Name
correlation		corrcoef(x,y)	cor(x,y)
mean		mean(x)	mean(x)
function	def func(x): print x return x		func <- function(x) { print(x) return(x) }
dot product		dot(x,b)	x*b
transpose		transpose(x)	t(x)
matrix product		b*x	t(b) %*% x
random	random.random()	random.rand(1)	runif(1,0,1)
sort	x.sort()		sort(x)
help	help(command)	help(command)	help(command), ??command

Who we reward

2011-03-23T09:50:00.000Z

Why do we reward people like Silvio Berlusconi or Charlie Sheen with our attention. They are bad at what they are supposed to do. They are addicts. You don't have to be moral about it, they are simply ugly.

Why don't we celebrate (more) people who create something beautiful, practical or useful to society?

Maths of self-publishing

2011-03-22T09:38:00.000Z

Joe Konrath has decided to self-publish rather than accept a 500k advance. He explains that he would get 70% rather than 14.9%. He also explains that pricing books cheaper, will get him higher e-sales. In the table below I have calculated that with these figures he only needs to sell 43% of what the publisher expects to sell to make the same money.

http://jakonrath.blogspot.com/2011/03/ebooks-and-self-publishing-dialog.html

	Price	Royalty %	Sales	Royalty
w/ publisher	$9.99	14.9%	335,906	$500,000.00
self pub	$4.99	70.0%	143,143	$500,000.00
%	50%	470%	43%

Groupon

2011-01-13T09:47:00.000Z

Everyone is talking about Groupon, I wanted to check how fast they are growing. The below shows the global visitors - Groupon has overtaken Yelp but is closely followed by Living Social. It looks as though Living Social could actually overtake Groupon. I am not sure if the dip is due to the incomplete January.

How to hack Google charts

2011-01-13T09:38:00.001Z

Kaggle social network challenge - test/train code

2011-01-12T09:28:00.000Z

For those having participated in the Kaggle social network challenge here is the Python code to split the full downloaded graph into test and training.

#create random sorted train set and test set with equal amounts of true and false edges

import random

samp=9000

#import complete file
f1=open('complete4.txt','r')
f2=open('simplesplit_test.txt','w')
f3=open('simplesplit_validate.txt','w')
f4=open('simplesplit_train.txt','w')

prim=[]
prim_set=set()
sec_set=set()
prim_connections={}
prim_2plus=0
sec_connections={}
sec_2plus=0
for line in f1:
    a=line.split(',')[0]
    b=line.split(',')[1].strip()
    prim.append([a,b,random.random()]) #need rand for later
    if a in prim_set: #if seen before
        prim_connections[a]+=1
    else:
        prim_connections[a]=1
    if b in sec_set: #if seen before
        sec_connections[b]+=1
    else:
        sec_connections[b]=1
    prim_set.add(a)
    sec_set.add(b)

print len(prim),len(prim_connections),len(sec_connections)

#universe of those with 2+ connections
prim_universe=set()
for p in prim_connections.keys():
    if prim_connections[p]>1:
        prim_2plus+=1
        prim_universe.add(p)

#universe of those with 2+ connections
sec_universe=set()
for p in sec_connections.keys():
    if sec_connections[p]>1:
        sec_2plus+=1
        sec_universe.add(p)

print prim_2plus,sec_2plus

#chose 2 sets 5000
sample=random.sample(prim_universe,samp)
sample1=set(random.sample(sample,samp/2))
sample2=set([i for i in sample if i not in sample1])

print len(sample),len(sample1),len(sample2)

#sort by random
prim2=sorted(prim,key=lambda rand:rand[2])

del prim

prim3=[]
sample1_done=set()
for i in prim2:
    if i[0] in sample1:
        if i[0] not in sample1_done and (sec_connections[i[1]]>1 or i[1] in prim_connections): #not done and inbound has other edge
            sec_connections[i[1]]-=1
            f2.write(i[0]+','+i[1]+'\n') #test
            f3.write(i[0]+','+i[1]+',1\n') #validate
            sample1_done.add(i[0]) #is done
            print len(sample1_done)
        else:
            f4.write(i[0]+','+i[1]+'\n') #train
    else:
        f4.write(i[0]+','+i[1]+'\n') #train
        if i[0] in sample2: #create a subset of prim to speed up non pairs check
            prim3.append([i[0],i[1]])

del prim2

print len(prim3)

#for sample2 chose non connections
count=0
for i in sample2:
    if count
        done=0
        prim4=[j[1] for j in prim3 if i==j[0]] #a subset
        while done==0:
            rand=random.sample(sec_universe,1)[0] #because 1 returns set
            if rand not in prim4 and rand<>i:
                done=1
                count+=1
        print count
        f2.write(i+','+rand+'\n') #test
        f3.write(i+','+rand+',0\n') #validate
    else:
        break

f1.close()
f2.close()
f3.close()
f4.close()

Android lock

2011-01-11T22:03:00.000Z

I have tried to determine the number of possible combinations on a 3x3 Android pattern lock. I come up with 10,305 combinations which is slightly more than the 10,000 combinations you would get with a 4 number lock. Let me know if you find any errors.

#count how many patterns there are on 3x3 lock
#every point visited once
#can move straight and diagonal
#path length 1 to 9

#length 1 is trivial: 9 possibilities

done=dict()
for c in range(1,4):
    for r in range(1,4):
        poss=len(done)
        done[poss]=[[c,r]]

poss=len(done)
print poss

def posspath(curpath): #returns all possible paths from a curpath
    outpath=[]
    #can go 8 different ways
    for add in ([0,1],[1,0],[1,1],[0,-1],[-1,0],[1,-1],[-1,1],[-1,-1]):
        c=curpath[-1][0]
        r=curpath[-1][1]
        #if within bounds and not visited
        if 1<=c+add[0]<=3 and 1<=r+add[1]<=3 and [c+add[0],r+add[1]] not in curpath:
            outpath+=[curpath+[[c+add[0],r+add[1]]]]
    return outpath

for path in range(2,10):
    for c in range(1,4):
        for r in range(1,4):
            curpath=[[c,r]]
            nextpath=posspath(curpath)
            i=0
            while i
                if len(nextpath[i])==path:
                    if nextpath[i] not in done.keys():
                        poss=len(done)
                        done[poss]=nextpath[i]
                else:
                    #explore possible and remove original
                    nextpath2=posspath(nextpath[i])
                    nextpath.remove(nextpath[i])
                    nextpath+=nextpath2
                    i-=1
                i+=1

poss=len(done)
print poss

Pop influencers

2010-11-06T15:27:00.000Z

I did another graph similar to the jazz one, starting with Lady Gaga. Again I just keep influencers (indegree>=1) and get this network.

Jazz network

2010-11-06T14:51:00.001Z

I have written a Python script which scrapes 'influenced by' from allmusic.com. My start was one of the greatest jazz musicians of all time: John Coltrane. I then put it into a DL file and visualised it, the node size is set by indegree (ie those who are quoted as influencers). Louis Armstrong and Coleman Hawkins are key influencers. The disadvantage of this is that it usually goes back in time, so if you started with a younger jazz musician you should capture more of the golden years of jazz 1960s.

Here is the code: https://docs.google.com/leaf?id=0B_YKhy3eG-qJYmYwZmU1NjEtMWI0OC00ZDFhLWJhOWUtMjQ3MWU0ZDg0ODIx

Appassionata

2010-11-02T21:53:00.000Z

Here is a Spotify playlist of Lang Lang, Richter, Gould and some others playing Beethoven's Appassionata. Gould plays it the slowest, whereas Serebriakov plays it the fastest.

Marketing

2010-11-01T20:52:00.000Z

Save you marriage - have an affair

2010-11-01T08:28:00.000Z

I got served both these ads, aren't they contradicting each other, or one leads to the other?

Relevant ads

2010-10-31T15:03:00.000Z

I am getting Spanish ads from the US Dept of Agriculture. They are spending their money wisely.

Mahler's Symphonies

2010-10-08T12:38:00.000+01:00

I have assembled Lebrecht's list of great Mahler symphony recordings in one Spotify list:
Mahler Lebrecht

Rock Paper Scissors bot

2010-09-23T13:06:00.000+01:00

I read this post http://flowingdata.com/2010/07/30/how-to-win-rock-paper-scissors-every-time/ and wrote some Python code which uses 2 of the hints given in the diagram. This bot will win 57% of the games.

I simulate a human draw using the % given and hence the bot never draws scissors unless it's a response to a 2 in a row from the human player.

#simulate a human player
#and answer with best strategy

import random

games=1000
wins=0
draws=0
hist=[]

def humandraw(hist):
    rand=random.random()
    if rand<.296: d='P'
    elif rand<.296+.354: d='R'
    else: d='S'
    if len(hist)>1:
        if hist[-1]==hist[-2]:
            while d==hist[-1]:
                rand=random.random()
                if rand<.296: d='P'
                elif rand<.296+.354: d='R'
                else: d='S'
    return d

def winner(p1,p2):
    if p1==p2: win=0
    elif p1=='R' and p2=='S': win=1
    elif p1=='S' and p2=='R': win=2
    elif p1=='P' and p2=='R': win=1
    elif p1=='R' and p2=='P': win=2
    elif p1=='S' and p2=='P': win=1
    elif p1=='P' and p2=='S': win=2
    return win

for g in range(games):
    d=humandraw(hist)
    rand=random.random()
    if rand<.5: a='P'
    else: a='R'
    if len(hist)>1:
        if hist[-1]==hist[-2]:
            if hist[-1]=='P': #will draw S or R
                if rand<.5: a='R'
                else: a='P'
            elif hist[-1]=='R': #will draw P or S
                if rand<.5: a='R'
                else: a='S'
            elif hist[-1]=='S': #will draw P or R
                if rand<.5: a='S'
                else: a='P'
    win=winner(a,d)
    #print a,d,win
    if win==1: wins+=1
    elif win==0: draws+=1
    hist.append(d)

print games,wins,draws
print float(wins)/(games-draws)

Google's new data structure

2010-09-13T10:00:00.000+01:00

Google Caffeine — the remodeled search infrastructure rolled out across Google's worldwide data center network earlier this year — is not based on MapReduce, the distributed number-crunching platform that famously underpinned the company's previous indexing system. As the likes of Yahoo!, Facebook, and Microsoft work to duplicate MapReduce through the open source Hadoop project, Google is moving on.

http://www.theregister.co.uk/2010/09/09/google_caffeine_explained/

Cab fares in London

2010-08-14T11:28:00.000+01:00

I have done some calculations on the back of the TfL examples given on their websites and the median price per mile (excluding the £2.20 minimum) during the day is £3.44 per mile. TfL give you the fare range so I have used the mid fare.

You can make your estimates using this table or using the Excel in the link below.


Distance	Approx journey time	Monday to Friday 06:00 - 20:00 (Tariff code 1)	Mid price	per mile	Monday to Friday 20:00 - 22:00 Saturday and Sunday 06:00 - 22:00 (Tariff code 2)	Mid price	per mile	Every night 22:00 - 06:00 Public holidays (Tariff code 3)	Mid price	per mile
1 mile	5 - 12 mins	£4.60 - £8.40	£6.50	£4.30	£5.00 - £8.60	£6.80	£4.60	£5.20 - £8.60	£6.90	£4.70
2 miles	8 - 15 mins	£7.20 - £11.20	£9.20	£3.50	£7.20 - £11.20	£9.20	£3.50	£8.00 - £12.40	£10.20	£4.00
4 miles	15 - 30 mins	£11 - £19	£15.00	£3.20	£13 - £19	£16.00	£3.45	£15 - £23	£19.00	£4.20
6 miles	20 - 40 mins	£17 - £28	£22.50	£3.38	£19 - £28	£23.50	£3.55	£24 - £34	£29.00	£4.47
Between Heathrow and Central London	30 - 60 mins	£43 - £75			£43 - £75			£43 - £75
Avg per mile				£3.44			£3.53			£4.33

Miles	19			£68.00			£69.00			£85.00

http://spreadsheets.google.com/ccc?key=0AvYKhy3eG-qJdDFMVHN2MV84a2Y5blpjUi1QMEV3VUE&hl=en_GB

Email broadcast checklist

2010-07-27T20:31:00.000+01:00

Here is a checklist of what to ask before you send out an email campaign

Are your target counts right?
Do you have text and HTML version?
Are your pictures uploaded?
Do you have mirror and unsubscribe links?
Do you have the right ratio of copy and pictures?
Do you avoid spam keywords?
Is tracking set up?
Have you got an auto-reply for the reply-to address?
Is personalisation set up?
Have you taken steps not to be blacklisted? http://www.mxtoolbox.com/blacklists.aspx

Lines exe

2010-06-29T14:45:00.000+01:00

The exe in the linked zip file prints the first or last rows of a very large file. You need to extract all files into the same folder as your large file, then in DOS run

lines [your file]

This will print 10 rows and also count number of rows.
You can change how many rows you see and if it's top or bottom like

lines  -l 20 -t B [your file]

where 20 is number of rows and B is bottom.

If you want the output directed to a new file do

lines  [your file] > [new file]

Government organisations by staff cost (salary)

2010-06-25T15:46:00.000+01:00

I have had a look at the workforce cost data release. I have calculated a cost per FTE and taken those with more than 100 FTE. The costs seem rather high and might include expenses and pension contributions I guess. Nevertheless ODA and NDA get the most. The highest ranking proper department is International Development.

Organisation name	Main/parent/sponsoring department (where applicable)	Cost per FTE
OLYMPIC DELIVERY AUTHORITY	DCMS	£102,369.67
Nuclear Decommissioning Authority	DECC	£94,634.83
Partnerships for Schools	Department of Education	£86,021.51
Government Actuary's Department	Government Actuary's Department	£79,230.77
Personal Accounts Delivery Authority	Department for Work and Pensions	£72,138.96
Office of Rail Regulation	DfT	£64,484.19
Department for International Development	Department for International Development	£64,410.68
NS&I	H M Treasury	£63,750.32
SEEDA	BIS	£63,739.65
Cabinet Office	Cabinet Office	£63,322.03
OFFICE OF GAS AND ELECTRICITY MARKETS	OFFICE OF GAS AND ELECTRICITY MARKETS	£62,942.78
The Pensions Regulator	DWP	£60,993.98
Office of Government Commerce	H M Treasury	£60,389.45
Audit Commission	Communities and Local Government	£59,457.83
Pension Protection Fund	DWP	£59,306.57
Young People's Learning Agency	DFE	£59,174.88
Tenant Services Authority	Communities and Local Government	£57,858.23
Dept. of Health (core)	Dept. of Health	£57,550.96
Treasury Solicitors Department	Treasury Solicitors Department	£57,225.06
Advantage West Midlands	BIS	£57,012.42
SOCA (including CEOP)	Home Office	£56,919.73
CLG - Central Department	Communities and Local Government	£56,716.60
Export Credits Guarantee Department	Export Credits Guarantee Department	£56,398.10
Core DECC	Department of Energy and Climate Change	£56,393.67
Department for Culture, Media and Sport	Department for Culture, Media and Sport	£56,235.54

Top 10 government websites

2010-06-25T15:28:00.000+01:00

Here are the top 10 government websites by number of unique users in March 2010. The first six are all service driven.

DoH and Justice actually attract more visitors than No10. The OPSI page is a bit of a surprise but it seems to act as host.

Website	Unique Users Mar-10
direct.gov.uk	11,433,948
nhs.uk	6,975,509
hmrc.gov.uk	4,360,065
businesslink.gov.uk	1,715,043
opsi.gov.uk	1,501,135
transportdirect.info	1,447,337
dh.gov.uk	1,038,601
justice.gov.uk	1,003,197
number10.gov.uk	879,726
ofsted.gov.uk	800,676

Data is here

Structured v unstructured data

2010-06-12T23:21:00.000+01:00

Is the data on the web unstrcutured? It is to some extent, especially
what is said by individuals (buzz). The rest should be pretty
structured because you can exploit the HTML/PHP/XML syntax. With the
advent of the semantic web it is becoming more structured. The web is
becoming more like a neural net.

Crisis and employment

2010-06-12T23:03:00.001+01:00

The UK job market has not been hit as hard by the economic crisis as
some expected. On the other hand the upturn in the US has mainly been
jobless. I think what has happened is that companies have streamlined
to increase productivity. In the UK people have been put on shorter
work rather than being fired. In the US firing is easier. This crisis
will exacerbate the underlying trend for unskilled jobs. We need a
educational revolution in the West not a scaling back as suggested by
the Lib Tory coalition, to save our society from further splitting
apart.

Giving away email addresses

2010-05-24T14:35:00.001+01:00

Many people are aware that they increase spam by posting their email address online, they try to circumvent that by writing their email like me at gmail dot com. But the Python code below shows how to retrieve the email address from a string (even if people use brackets etc). So be cautious.

                text2=text.lower()
                text2=text2.replace(' dot ','.')
                text2=text2.replace(' (dot) ','.')
                text2=text2.replace('(dot)','.')
                text2=text2.replace(' [dot] ','.')
                text2=text2.replace('[dot]','.')
                text2=text2.replace(' {dot} ','.')
                text2=text2.replace('{dot}','.')
                text2=text2.replace(' at ','@')
                text2=text2.replace(' (at) ','@')
                text2=text2.replace('(at)','@')
                text2=text2.replace(' [at] ','@')
                text2=text2.replace('[at]','@')
                text2=text2.replace(' {at} ','@')
                text2=text2.replace('{at}','@')
                #now put into list
                text3=text2.split()
                for w in text3:
                    if w.find('@')>0 and w.find('.')>0:
                        if w.rfind('@') is not w.find('@'): #in case of 2 at
                            email=w[w.find('@')+1:]
                        else:
                            email=w

Google Social Graph in Python

2010-05-21T20:24:00.000+01:00

I have started using the Google Social Graph API. I am running it in Python. It's pretty straightforward. I am using the edges out option (edo=1) and then collect all nodes_referenced. I do this up to the fourth degree. This way you can check what information about you or your friends is out there and can be linked up.

socialgraph.py

Twitter Follower Hack Has Twitter Leaping to Fix, Tweeps Panic-Tweeting [Updated] - via Fast Company

2010-05-11T16:23:00.001+01:00

"Want to force anyone on Twitter to follow you (yes, even the wondrous
Mr. Fry)? There's a hack for that, and it's damned simple. The thing
is, it looks like it's kinda, sorta, maybe broken Twitter...and
everyone has zero followers.

Gizmodo just had a piece demonstrating the hack, which they speculate
may be a layover code from Twitter's earlier days that's still in
action. It couldn't be simpler: Visit Twitter.com, log in to your
profile, click on "Find people" and in the search box type "accept
xxxxxx" replacing the x's with the username of your desired target. It
may throw up an error message, but it seems to work very reliably.

The act has been quickly christened Twape on the Intertubes, meaning
"Twitter rape" and it's potentially incredibly important. So
incredibly important, in fact, that the community is freaking out
(while occasionally throwing out good jokes,) and not only because of
Twape itself, but because it looks at first glance like it may have
broken the entire Twitter system. But really this is probably a sign
that Twitter may already be on the case--if you visit Twitter.com
right now you'll see your follower counts are at zero...though your
feed will still connect to all the right Tweeps that you were
originally following.

This, folks, is a living act to demonstrate exactly how powerful
Twitter has become. For users who make the most of Twitter's global
reach to promote their wares, connect with customers or engage in
dialog with clients and friends, the list of people following you is a
jealously monitored and important thing (even while research
repeatedly shows that it's not how many people who follow you, but who
is following you that counts.) Any notion that Twitter may somehow
lose track of this data is frightening. Fingers crossed, hey, Tweeps?

Update: Twitter's confirmed that it's on the case, and that user's
follower counts will be returning to normal. A quick scan of the feeds
shows that this is only true for a limited number of Tweeps for the
mo, but we have confidence in the system."

via Fast Company

I am six users

2010-05-11T07:07:00.001+01:00

I am browsing the Internet at work using two browsers, on the iPhone, on the blackberry, and at home using two browsers. This means web analytics packages would count me six times if I visited the same page from all. Are the counts over-inflated?

Let last.fm recommend songs

2010-04-27T16:38:00.001+01:00

Even if you are not subscribed to last.fm you can use their API to get track recommendations, replace the artist and track in this URL and you get a XML list of similar songs http://ws.audioscrobbler.com/2.0/?method=track.getsimilar&artist=cher&track=believe&api_key=b25b959554ed76058ac220b7b2e0a026 (use the sample key)

Another example http://ws.audioscrobbler.com/2.0/?method=track.getsimilar&artist=gipsy%20kings&track=un%20amor&api_key=b25b959554ed76058ac220b7b2e0a026

View slideshows

2010-04-25T20:25:00.000+01:00

Replace 'test' in http://www.flickr.com/search/show/?q=test&l=cc&ct=0&mt=photos&adv=1 and see a slideshow, this example skips videos and shows only CC content, so in case you like something you can pause and copy the picture.

Tory and Labour manifesto speeches

2010-04-14T21:06:00.000+01:00

Here are word clouds of the 2 manifesto sepeechees by the Cameron and Brown (could not locate Clegg's). As you can see Cameron talks a lot about the government, people, power and together. Brown talks about Britain, people, future. The word fair doesn't actually come through that much in Brown's speech.

Trombone bumblebee

2010-04-10T17:46:00.002+01:00

Internet spring clean

2010-04-10T17:31:00.001+01:00

Help clean the internet

update or delete old sites
delete old blogs
delete social network accounts you don't use
fix broken links
reduce spam
unsubscribe from useless lists

Dirk

2010-04-10T17:31:00.000+01:00

(BN) Soros Plans Oxford University Economics Institute on Markets, Times Says

2010-04-05T17:34:00.000+01:00

Bloomberg News, sent from my iPhone.

Soros Plans Economics Institute at Oxford University, Times Says

April 5 (Bloomberg) -- George Soros, the billionaire U.S. investor, is helping to establish an economics institute at Britain's Oxford University, the London-based Times reported.

The institute, to be headed by Professor David Hendry, a fellow of Nuffield College, will be part of the university's James Martin 21st Century School; Soros and the school itself are each contributing $5 million, the newspaper said.

The initiative is part of a campaign by Soros to push the economics profession away from the idea that markets should be left to themselves, a premise that, he believes, helped to cause the global financial and economic crisis, the Times said.

Soros hopes to set up further institutes at universities in Germany, France, China, Italy and the U.S., the newspaper said, adding that Oxford was picked for the first because the financier sees Britain as more open to fresh economic thinking than the U.S.

Soros's funding is being channeled through the New York-based Institute for New Economic Thinking, or Inet, established last year; Robert Johnson, a former managing director at Soros Fund Management LLC who now heads Inet, said the economic crisis and economists' failure to predict it show that a broader approach to economics is necessary, taking in history, psychology, natural science and even literature, the Times reported.

Click here for web link

Find out more about Bloomberg for iPhone: http://m.bloomberg.com/iphone

Dirk

Google public data

2010-03-27T13:29:00.000Z

Google has started its own data tool. It's still quite basic and has mainly EU and US stats but it's a nice addition. Here it shows that the UK which used to have always lower unemployment, has now higher unemployment than Germany.

http://bit.ly/9QtdW7

This week

2010-03-27T13:25:00.000Z

Diane Abbott, who appears on BCC1 This Week, is getting increasingly annoying. She interrupts guests when she doesn't agree and has an awful temper. Her dress sense which was never great also seems to diminish. I did actually like her once but she seems to be a really difficult person.

Movies of the Decade

2010-03-22T20:39:00.000Z

You can browse the movies of the decades on IMDB by replacing the number in the following URL http://www.imdb.com/search/title?year=1940,1949&title_type=feature&num_votes=10000,&sort=user_rating,desc

40s Casablanca
50s 12 Angry Men
60s Good, Bad, Ugly
70s Godfather
etc

SAS search macro

2010-03-22T13:32:00.002Z

I have written a search macro in SAS which performs text search on various variables. It allows for and/or logic although only one at a time. Let me know what you think. Can be downloaded form Google Docs.

%search(dset, search data set
    str, characters to search for
    vars, variables to search in
    or=N,    N=and Y=or
    case=N, Y=case sensitive
    word=Y, Y=look for word N=look for words containing
    print=Y Y=prints
    out=search output data set
)

Usage
%search(one,dirk nachbar,firstname lastname) searches for dirk AND nachbar (not case sensitive) in variables firstname
and lastname in table one

http://docs.google.com/leaf?id=0B_YKhy3eG-qJYjE5NWFkYWUtMjUzNi00N2RkLWIyZjItNjEzZmMzNDMyYTQ5&hl=en_GB

Curb your Enthusiasm favourite - Larry acts Jewish

2010-02-14T20:23:00.000Z

one of my favourite ads

2010-01-23T08:05:00.002Z

Class and income

2010-01-22T10:17:00.002Z

There is a lot of talk about Labour making the election about class – probably to counter the Tories' Eton image. I think they are wrong in many ways. Firstly, they equate class and income, which is nonsense. You can be friends with royalty and not be rich and vice versa. In fact a lot of entrepreneurs didn't go to private school or university – and they and their children live better than many. Secondly, Labour's record on income inequality is so poor that they shouldn't even mention this. If they couldn't fix it in 13 years, how are they going to do it in the next 4 years when less growth is there to be distributed?

Information regarding Ocado deliveries from 18th January - bad comms

2010-01-18T11:53:00.001Z

Ocado are usually good at customer communications. But this one really confused me. The subject states that it's about changes form 18th Jan, but this only gets mentioned in paragraph 4 where they say that I won't get a free newspaper anymore on some rubbish green pretext - which they lay out in paragraphs 1 to 3. Come on Ocado, you can do better than that.

Having trouble reading this email? View the web version

Dear Mr Nachbar,

As an efficient and environmentally friendly way of getting groceries from field to kitchen table, it's fair to say we're way out in front. Perhaps it's no big surprise: after all, it's something we've been working on since our very first delivery back in 2002.

But "being green" isn't just the latest marketing buzzword to us. It's at the very heart of what makes us Ocado, and no-one else. Like painstakingly paring down the weight of our van to enable us to deliver to more people using the same amount of fuel. Or completely overhauling our ordering system to reduce our food waste (at just 0.3% of total sales, we've got good reason to believe that we're industry leaders - and we're not finished yet). Or even becoming the first UK supermarket to actively collect and remake its own grocery bags.

These are some of the ways we've become more efficient, but there's still more to do. As founder members of the 10:10 campaign, we've pledged to reduce our carbon footprint by 10% this year. Making our grocery bags even more environmentally friendly will help us get there. So, too, will our trials of an innovative low-emission electric delivery van.

On a separate note, we've also been talking to our friends at The Times, and we've made a joint decision to stop including a free newspaper with Ocado deliveries from 18th January 2010. We're looking at continuing our partnership in other ways, however; and, to begin with, we're offering you a free seven-day trial of the electronic version of The Times.

To sign up for your free seven-day trial, simply follow this link to The Times' website and enter the code TIMES_TRIAL2.

Here's to the start of another green year at Ocado!

Jon Rudoe
On behalf of the Ocado team

Voted favourite online supermarket in Which? magazine 2009 reader survey

Online Retailer of the Year 2005, 2007 & 2009, The Grocer Gold Awards.

Green Retailer of the Year 2009, The Grocer Gold Awards.
These prestigious awards recognise our efforts to deliver a more convenient,more sustainable alternative to supermarket shopping.

Large Retailer of the Year 2008, Online Green Awards.
We won this award in recognition of our revolutionary green approach to selling groceries

To help stop Ocado emails being seen as spam, please add 'ocado@ocadomail.com' to your address book.
If you would prefer not to receive any more information from Ocado please unsubscribe from our emails at any time.
The registered company address of Ocado Limited is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts, AL10 9NE.
Registered in England. Company registration number: 3875000

JavaScript hack enables Flash on iPhone

2010-01-15T15:51:00.002Z

A rather clever programmer has managed to get the iPhone to run interactive apps created using Adobe's Flash platform. And because it works inside the Safari browser, it isn't subject to the dictatorial rules of Apple's App Store.

The software is called Gordon and it doesn't actually allow Flash to run natively on the iPhone. Instead, Gordon is a JavaScript runtime, written by Tobias Schneider, which enables the browser to play and display Flash files. A runtime is a collection of software that allows the running of code inside it. A helpful analogy is a software emulator for a games console. These enable you to play the original code of, say, Super Mario World, but inside an application that runs on your PC. It's also how games such as Sonic The Hedgehog are able to run on your iPhone.

View article...

Juicy Leak: Orange France's Boss Basically Confirms the iSlate [Updated]

2010-01-11T17:20:00.001Z

Oh boy, adding to the spiraling whirl of Apple Tablet rumors it looks like the number two exec at cell-phone network Orange France has basically confirmed the iSlate is real, it's coming soon, and it'll have a global launch.

The news is coming via an interview given by Stéphane Richard on Europe 1 this morning. Richard is a senior exec at France Telecom/Orange, and as part of a longer discussion he was thrown this sudden question: "According to the weekly Le Point, in a short while your partner Apple is going to launch a tablet..." This could easily have been dismissed by Richard as idle speculation masquerading as a leading question. But Richard simply responded "oui..."

And thus came the next question, again speculative, but definitely mining for the right kind of info: "...with a Webcam..." Which got another "Oui" from Richard. And then came the interesting bit:

Interviewer: "And Orange users will be able to benefit from this too?"

Richard: "Of course!"

Basically, given three opportunities to either deflect the questions by saying "well, nobody knows..." or flatly deny the speculation, or even to squirm out of a direct answer like a politician, Richard chose not to. Or he forgot not to. Or he was allowed to leak some info by Apple. He was even enthusiastic about the matter--though note that, despite some of the mis-translated excitement about this elsewhere online, Richard didn't agree it was due in a "couple of days", instead that "quelques jours" (which aligns more with "someday soon") supports more the January announcement/March launch rumors we've been hearing recently.

Why should we pay attention to Richard's words? Because Orange is a senior player in the global cell-phone game--it controls vast grids in Europe and Africa, and with all its subsidiaries lumped together it's actually the world's fifth biggest operator. It's also a key iPhone distributor, and one that was chosen by Apple for early iPhone love before the multicarrier model really took off in Euroland.

Admittedly these are just off-the-cuff words. But if you add them to similar-feeling confidently assertive statements by The New York Time's Bill Keller back in October (noting that Keller even called it the "Apple slate") and the slew of detailed leaks that seem to be popping up at the moment, it really is going to get gadget fan's blood pumping. Particularly exciting is the fact that this concerns an imminent arrival of the gizmo in France--suggesting an international launch at the same time, which is different to the strategy Apple employed for the iPhone.

[Via Nowhereelse.fr]

Update: Here's the video feed for your interest, in French of course.

It's clear from the video that Richard is a little off-balanced by these questions, and you could perhaps assume he's speaking as speculatively as the interviewer himself. But he definitely mentions video calling, and the necessary infrastructure changes to support the increased data load, and that rules out that he's talking about iPhones.

View article...

Enclosures:

apple-orange.jpg (11 KB)
http://www.fastcompany.com/files/imagecache/listing_image/files/apple-orange.jpg

Google retro

2010-01-09T07:03:00.000Z

Here you can find old Google logos from 1999 http://web.archive.org/web/19990224043535/www.google.com/stickers.html

What Would Google Do?

Failed bomber

2010-01-08T08:17:00.002Z

The failed bomb attempt on Christmas day and the failure of the security services is quite interesting from a data point of view. Apparently the agencies gave different databases which were not linked up properly. This is almost like in a badly run marketing or sales department. What the agencies need is a single suspect view. However the risk of errors is amazingly higher than in any marketing department.

The History of Terrorism: From Antiquity to al Qaeda
Inside Terrorism

Never gonna give you up as world cloud

2010-01-04T15:06:00.000Z

Why Do I Need A Single Customer View

2010-01-04T15:02:00.003Z

Why Do I Need A Single Customer View

View more documents from Dirk Nachbar.

Customer Data Integration: Reaching a Single Version of the Truth (SAS Institute Inc.)

400th post: New Year and Health

2010-01-03T07:38:00.000Z

It is amazing to see how many people are jogging on the first days of January and indeed Google Insight shows a peak in dieters and runners (inc gym). This healthy trend stays there until roughly late summer. The marathon is a big thing people search for in April (London Marathon), but relative interest is decreasing.

Source:
http://www.google.com/insights/search/#q=diet%2B%22lose%20weight%22%2B%22weight%20loss%22%2Cjogging%2Brunning%2Bgym%2Cmarathon&geo=GB&cmpt=q

Alli Weight-Loss Aid, Orlistat 60mg Capsules, 120-Count Refill Pack
The UltraSimple Diet: Kick-Start Your Metabolism and Safely Lose Up to 10 Pounds in 7 Days
Runner's World Complete Book of Women's Running: The Best Advice to Get Started, Stay Motivated, Lose Weight, Run Injury-Free, Be Safe, and Train for Any Distance (Runner's World Complete Books)

On demand players

2009-12-23T19:42:00.004Z

BBC has just released figures showing Top Gear being top of the iplayer list. However, Google Insight reveals that while the iplayer has been the leading on demand service over 2008, it has been surpassed by ITV player and 4oD this year (in terms of search). 4oD has actually been around the longest.

http://www.google.com/insights/search/#cat=3&q=iplayer%2C4od%2Citv%20player%2Con%20demand%2Cdemand%20five&geo=GB&cmpt=q

RATM vs McElderry follow up

2009-12-21T10:11:00.004Z

Google Insight was right and RATM did win first spot of the charts. My caution last time proved wrong, the dip from last time now actually disappeared because the trend was up even more.

Climb
Rage Against the Machine

RATM vs McElderry

2009-12-16T21:24:00.003Z

If you are interested to see who will win the war of the charts, Google Insight gives you some idea. Note that this only goes up to the 12th, RATM seems to have peaked and McElderry might actually win.

Dilbert

2009-12-15T08:13:00.000Z

Analystics or Econonetrics

2009-12-13T08:30:00.001Z

Analytics or Econometrics

View more presentations from Dirk Nachbar.

web trends

2009-12-10T20:01:00.003Z

I have trended some of the buzz words around the internet and new technologies in Google Insight. For about 3 to 4 years web 2.0 or 3.0 was the prevalent topic. However more recently social media and networks (social networks have always been there) are replacing this debate. It’s not so much about the capabilities but about the user interaction. AR is the latest upcomer which will grow big next year (Google is cautious about the forecast here).

Unique visitors

2009-12-04T17:03:00.002Z

Unique visitors is usually seen as key metric in web analytics. Visitors are tracked via a cookie or a IP number. The IP number method is seen as less reliable as some providers allow users to change their IP number. The safer cookie method has flaws as well because

·         People can turn cookies off or delete them
·         People can access the same website with different devices (home, work, mobile)
·         People can use RSS feeds to access content

I think the second point becomes more important now because with the rise of smart phones most people have 2 devices being able to access the internet. So next time you see unique visitors going up as yourself if that's really the case.

PS: There is another point why an increase in uniques is not significant, the increasing internet audience, this is why reach is actually a better measure.

Amex and Tesco

2009-12-03T13:50:00.001Z

There has been recent news that Amex wants to follow the Tesco Clubcard (dunnhumby) model and start selling aggregated data to whoever is willing to pay for it. Amex' global customer base of 90m makes Clubcard's 14m look small. However, it will be quite a challenge to filter out data or a segmentation which will appeal to all retailers and consumer goods companies. Amex has a very biased customer base, which is good for targeting (which I understand is not part of this because Amex customers never opted in) but not so good for market research. It would be great if Amex could go a step further and create a journey segmentation which would show how customers shop certain categories. I think it will take years for them to data mine their (90m times hundreds of transactions) database.

Fwd: Delivery Status Notification (Delay)

2009-12-02T16:57:00.001Z

I emailed my own Google wave account and received this error message. So does that mean wavers cannot communicate with others?

---------- Forwarded message ----------
From: Mail Delivery Subsystem <mailer-daemon@googlemail.com>
Date: 2009/12/2
Subject: Delivery Status Notification (Delay)
To: xxx@gmail.com

This is an automatically generated Delivery Status Notification

THIS IS A WARNING MESSAGE ONLY.

YOU DO NOT NEED TO RESEND YOUR MESSAGE.

Delivery to the following recipient has been delayed:

xxx@googlewave.com

Message will be retried for 2 more day(s)

Technical details of temporary failure:
Probe failed: Too many resources/connections in use for: googlewave.com

----- Original message -----

MIME-Version: 1.0
Received: by 10.223.75.136 with SMTP id y8mr913742faj.69.1259680916995; Tue,
01 Dec 2009 07:21:56 -0800 (PST)
Date: Tue, 1 Dec 2009 15:21:56 +0000
Message-ID: <2b2b3f430912010721x6e588ad6x2bf5a3cfeefd2a5b@mail.gmail.com>
Subject: test
From: Dirk Nachbar <xxx@googlemail.com>
To: xxx@googlewave.com
Content-Type: multipart/alternative; boundary=0015174c3c1c9680e20479ac513e

How do people search for restaurants

2009-11-21T08:38:00.002Z

In the UK, the generic restaurant guide seems to be in decline, Michelin is a famous brand and has peaks in autumn, toptable (reviews and booking) has become more prevalent.

In Germany there is a similar picture. The restaurant guide is in decline, Michelin leads before ADAC and Varta, but in contrast to the UK restaurant review/critique hasn’t quite picked up yet.

pay for news content?

2009-11-17T12:12:00.001Z

LONDON - The latest survey of consumers' attitudes to paying for online content shows that 48% of people in the UK would pay for news online and on mobile devices.

Boston Consulting Group surveyed 5,000 people in the UK, the US, Australia, France, Germany, Italy, Norway and Finland via Research Now's online research service.
The figure is markedly higher than a previous survey by PaidContent UK, in which just 5% of readers said they would pay to continue reading when asked "What would you do if your favourite news site began charging?".

The BCG survey asked "How much per month are you willing to pay to get online news on your PC or mobile?", including zero as one option in the multiple choice answer.

http://www.brandrepublic.com/DigitalAM/News/967141/Half-UK-US-readers-pay-online-news/?DCMP=EMC-Digital-AM-Bulletin

This is a really interesting difference since almost no-one will pay for what they get for free now, whereas half would pay given payment options. Given various payment options, respondents are less likely to chose no payment.

Ransome's Dock

2009-11-16T13:47:00.000Z

Restaurant in the docks of Battersea. Nice athmosphere and jazz music
in the background. The food could be better. The courgette fritter are
nice though slightly burnt. The fish and chips consisted of mainly
French fries and very little cod. The cod could have been seasoned
better. Nice service. 3/5

BNP online presence

2009-11-08T17:34:00.005Z

I am not supporting the BNP in any way.

The Question Time appearance of Nick Griffin has increased traffic to their site massively. In the weeks surrounding the show reach went as high as 0.3% of global traffic (alexa).

Interestingly their reach is higher than that of the three main political parties, even outside the QT period.

The profile from alexa shows that the visitors are mainly middle and retired men with either very low or high education (journalists).

Google Insight reveals that BNP has surpassed Labour as a search term as well. However not as much as the difference in reach - which suggests that many visit the BNP site directly.

Many BNP visitors also visit stormfront.org which is a US White supremacist forum.

my school

2009-10-31T20:06:00.002Z

here is a video (in German) of my school in Berlin - Carl von Ossietzky Gymnasium, where in 1988 a student got expelled by the Socialist government and had to go into exile in London

http://mediathek.ard.de/ard/servlet/content/1860624;jsessionid=076CAC937D33CD004EDCC8B4045BA345

Kraftwerk - word cloud

2009-10-25T08:57:00.002Z

German wiki

Digital Strangelove (or How I Learned To Stop Worrying And Love The Internet)

2009-10-24T10:24:00.000+01:00

Digital Strangelove (or How I Learned To Stop Worrying And Love The Internet)

View more documents from David Gillespie.

AI and 2nd life

2009-10-18T21:04:00.002+01:00

I am not using Second Life, but I came across the notion that it is artificial intelligence. Now I don't think it is because it's based on actual humans behind avatars creating characters, while the world seems to be fixed, is it? Would 2nd Life pass the Turing test?

Mando Diao - great song - not available in the UK

2009-10-04T19:04:00.000+01:00

Cameron at Marr

2009-10-04T09:57:00.001+01:00

Andrew Marr should relax. His attacking of Cameron on his show is
pathetic. Cameron is not in power and so he is not responsible. Marr
is very biased.

Sent from my iPhone

new job

2009-10-01T20:40:00.002+01:00

I have a new job now at targetbaseclaydonheeley.com - a digital agency.

Gran Torino

2009-09-13T14:02:00.003+01:00

I have just watched Gran Torino and I think it's an under-rated movie. It is extremely contemporary for many reasons

unemployment in post manufacturing US (I assume it was written before GM's collapse)
old age
racial inequality and brutality

The story is admittingly simple and the movie is carried by Eastwood who has tremendous economy in his acting. Walt's grumpiness and sarcasm is even funny. I also like how the movie plays with the Dirty Harry theme which is then chucked away at the end.

It deserved more awards than it received.

When should you check in your bags at the airport

2009-09-12T09:17:00.003+01:00

I have analysed this problem using 4 scenarios or quadrants. Let's assume it's a big flight and there is two cars which carry the bags from the terminal to the plane.

If you check in the 4th quadrant, then your bag goes to top of 2nd car B and you luggage is loaded as the third, it is unloaded as the second and goes on top of car A which brings the bag to baggage reclaim and the conveyor belt where it will be unloaded first. Reversely if you check in first you'll leave the destination last. This is all based on the first in last out principle and implies 'Take your time'.

Greenspan

2009-09-09T19:24:00.003+01:00

Greenspan's prediction that there will be more financial crises in the future is hardly noteworthy. For one his lax monetary policy caused some of the present crisis, and it is obvious that the capitalist system cannot escape the business cycle (boom and bust). His over-cautiousness now is part of the same animal spirits which caused the crisis in the first place - everyone's a bear now.

Polish power - German quality

2009-09-05T12:50:00.000+01:00

This is a slogan I saw at a building site in London.

It seems that the stereotype about Germans that they are thorough and
love detail and has helped me because people think this would apply to
me. Now in fact I am not so much into detail and live the big picture
more - I have a can do attitude which can be at odds with pedantry.

Sent from my iPhone

8 steps for a successful segmentation

2009-08-26T14:07:00.001+01:00

Get an overview of what data is available and at what level (how recent is the data)

Define the purpose of the segmentation (targeting, insight). Does it need to be updated?

Based on (2) define a success criterion (ie it separates pre-defined group A from group B)

Collect the data and normalise it, create an estimation sample

Correlate the data, can you eliminate certain variables (try variable clustering, factor analysis etc)

Start segmenting/clustering (note if you have very few variables maybe you prefer more rigid manual segmentation/classification)

Can you re-group certain segments?

Repeat until (3) is satisfied

Parameterise segmentation and roll out to rest of population; create a generic process to roll out as population changes

Review usage of segmentation after several months

Spotted

2009-08-07T08:32:00.000+01:00

British Iranian comedians prefer SW14. Saw Shappi Khorsandi and Omid
Djalili in our borough - separately that is.

Sent from my iPhone

MPs ignore public anger and give themselves 9,000 expenses deal by stealth - Telegraph mobile

2009-07-28T05:47:00.001+01:00

Just as bankers return to the good old times, the MPs are at it as
well. They have quietly increased the expense allowance they don't
need to account for with receipts. In this country nothing changes.
Thanks Labour.

http://m.telegraph.co.uk/;s=5coSD7o2tsLCF4v6zerAug02/article/5920660/

Sent from my iPhone

why is scientology advertising on youtube?

2009-07-13T22:18:00.002+01:00

next to Obama

what's in a doodle?

2009-07-10T21:44:00.000+01:00

http://foundmagazine.com/

Starbucks bingo

2009-07-09T22:51:00.003+01:00

Beim Italiaener

2009-07-08T21:45:00.001+01:00

2 and half year wasted

2009-06-30T21:27:00.004+01:00

The ONS has published new GDP figures today which show a large decline in GDP. People might think that four quarters of negative growth will be overcome once the economy starts growing again. In fact real GDP in Q1 2009 was as high as in Q3 2006. This means that the UK is a rich as war 2 and half years ago in real terms. And this neglects any population growth in that time.

BBC NEWS | Education | Child poverty 'get out' claim

2009-06-17T18:21:00.001+01:00

The new poverty legislation is a distraction from the fact that Labour
will miss its own PSA target to halve child poverty by 2010. In fact
in recent years no progress has been made.

http://news.bbc.co.uk/1/hi/education/8105030.stm

Sent from my iPhone

BBC getting very informal

2009-06-15T21:37:00.001+01:00

Journal of Targeting, Measurement and Analysis for Marketing

2009-06-12T19:11:00.002+01:00

See a free issue of the Journal of Targeting, Measurement and Analysis for Marketing

http://www.palgrave-journals.com/jt/journal/v16/n1/index.html

twitter

2009-06-01T21:40:00.000+01:00

Me on twitter http://twitter.com/dirknbr

Clive Granger dies

2009-05-29T16:54:00.000+01:00

The Nobel prize winning econometrician has died. He introduced the
ideas of cointregration and causality. Great loss.

Sent from my iPhone

Stats blog

2009-05-26T21:52:00.000+01:00

This is a great blog and site around understanding of risk: http://understandinguncertainty.org/davidsblog

Labour and child poverty

2009-05-19T20:38:00.003+01:00

It seems to have gone under in the turmoil around the expenses but the latest statistics show that Labour is way off its 2010 target of halving child poverty. The number of children in relative poverty before housing costs has stayed the same between 2006/7 and 2007/8 - at 2.9 million. This means only half a million lifted out of poverty since Labour came into power. Labour is not the party of equality.

http://www.dwp.gov.uk/asd/hbai/hbai2008/pdf_files/full_hbai09.pdf

Less than a tenth of a percent of delivered spam is clicked on

2009-05-06T23:04:00.003+01:00

MTA Delivery	82,700,000	21,100,000	10,100,000
User Site Visits	10,522	3,827	2,721
	0.01%	0.02%	0.03%

Kanich et al of Berkeley and UCSD in ‘Spamalytics: An Empirical Analysis of Spam Marketing Conversion’ have run a trial to calculate the spam conversion rates for three bogus campaigns. I have recalculated the rates based on emails actually delivered.

http://www-cse.ucsd.edu/~savage/papers/CCS08Conversion.pdf

Government internet database

2009-04-28T22:00:00.003+01:00

Maybe the u-turn on the internet database has also to do with the realisation what the cost of such a data base would be. Below I am making an estimate of how big an annual data set would be: 503 terabyte. This would need an army of IT specialists to maintain. A query on this data base would probably take a lot of time. And this is a conservative estimate assuming only the URL (and datetime) is kept and no search keywords or so. It also assumes emails are transferred into text and attachments are lost.

	Population	60,000,000
	Internet users %	73.00%
	Internet users	43,800,000
Websites	Sites per day	15
	Days per week	6
	Datetime field	8	byte
	URN field	8	byte
	Site field	100	byte
Emails	Emails per day	15
	Text size of email	1000	byte
	Email accounts per person	2


	Size of annual database	503,388,144,000,000	byte
		503,388,144,000	KB
		503,388,144	MB
		503,388	GB
	Size of monthly slice	41,949.01	GB

Small towns stereotypes

2009-04-15T18:20:00.000+01:00

People from small towns and villages are usually either very shy or
over confident because they are overwhelmed or think of themselves
masters of their little universe. Big city people are more likely to
be neurotic.

Sent from my iPhone

Strangling furious customers

2009-04-15T08:00:00.001+01:00

Nowadays every one has to be afraid when causing an argument about the
niggly rules at airports. They will probably arrest and question you.
All in the name of terror prevention. But have they ever seen a
terrorist complaining about bad customer service? It's just a means to
continue bad service and strangle the customer's rights.

Sent from my iPhone

Significance Importance Relevance

2009-04-09T20:40:00.003+01:00

You can think of these three concepts in a particular order. Significance means that something is possibly true. However, this only counts if it is actually important - has magnitude. Even if it is important, it's not guaranteed to be relevant. For it to be relevant it needs appeal and has to be practical/ implementable.

2009-04-08T20:29:00.001+01:00

I have uploaded a brief paper on testing for multi-modality using clustering - here.

In general modality is looked at from a distributional perspective. Silverman’s test uses a kernel density estimate (KDE) to test for modality. But the test statistic is biased and it can only test for one mode hypothesis against another. We propose a different way by looking at a distribution from the top.