提出了一個Markov Model，

可以同時將Document Clicks 和 Use Session 放進演算法中去cluster refinements。

例如query: Mars

通常Mars有兩種可能的走向：羅馬神話（戰神）、太陽系（火星）

這篇的重點在於將Query Suggestion cluster起來

讓Query Suggestion可以有足夠的Diversity讓user選擇

這篇的Motivation有三個：

• 1. 提供一個夠廣(diversity)的query refinements給user選擇
• 2. 提昇query suggestion顯示的方式
• 3. 能夠橫跨sessions去推薦query
• 4. 提供一個所有可能的diversity的summary給user選擇

針對第一點，目前的query refinements通常是clicks frequency導向

若某個面相的query refinements的frequency被兩個以上的query分掉了

那麼這個面相就很可能不會出現在query refinements中

第二點則是希望能夠透過column和row的方式顯示不同面相的query refinement cluster

讓每個column是不同的方向

幫助使用者了解目前的選擇

第三點則是透過第一次、第二次的query refinement去了解user目前的search intent

讓Search Engine可以更準確的推薦該方向的query refinement

現今有兩種方式做到query refinement clustering

一種是based on similarity in the vocabulary of the clicked documents

問題在於若兩種同一面相但長得完全不一樣的字就很難記算出similarity

(例如query Mars，若refinements是venus和jupiter，很難透過similarity將venus和jupiter cluster到星球這個cluster內）

一種是based on their occurrences within user search sessions.

然而user session常常會過於sparse或甚至有intention drift的現象

因此這篇paper提出一個combine document similarity和user session的Markov Model

並透過random walk去算出不同cluster

query會有時間性，會有spike(突然崛起)

這時候這些query跟document的關係可能就會改變

例如：query 地震

平常時候地震的相關document大都指到wikipedia

但是當地震發生的當下（有spike時），卻是指到New York Times（新聞媒體）

例如：March madness與NCAA的關係

平常NACC包含所有運動種類

但在三月時March madness會與NACC的basketball關係變得很強烈

因此這篇paper著重於觀察query spike的種類（馬上起來緩緩下降 or 緩緩上升急速下降…etc）

與document改變的量（query在不同時間下出現在document內的次數）

並透過這兩個特性去model user search intent

Fortunately I found a way from stackoverflow to use vim autocmd to create a python template automatically when I use vim to start a new python file.

Step 1: Add autocmd to your ~/.vimrc file to be triggered when you create a *.py file:

autocmd bufnewfile *.py :0r ~/.vim/template/pythontmp.txt
if exists('*strftime')
    au BufNewFile *.py :call append(2, '# Created: '.strftime('%a, %d %b %Y %T %z'))
endif

Step 2: Build your own python template in ~/.vim/template/pythontmp.txt

#!/usr/bin/env python
# -*-*- encoding: utf-8 -*-*-
 
"""
Document here.
"""
 
__author__ = u'username@xxx.xxx (ur_name)'
 
import sys
from optparse import OptionParser
 
def main():
    MSG_USAGE = "%prog [options] arg1"
    opt_parser = OptionParser(MSG_USAGE)
 
    if (len(sys.argv) != 2):
        print "[ERROR] unsupported arguments"
        opt_parser.print_usage()
        return 2
 
 
    return 0
 
if __name__=="__main__":
    sys.exit(main())

Step 3: Enjoy it.

Ref: http://stackoverflow.com/questions/6366686/add-creation-modified-time-to-python-template-in-vim-script

The way to sort a dictionary by value is as following:

import operator
x = {'a':1, 'b':3, 'c':2}
sorted_x = sorted(x.iteritems(), key=operator.itemgetter(1))
print sorted_x

and the results is

>>> import operator
>>> x = {'a':1, 'b':3, 'c':2}
>>> sorted_x = sorted(x.iteritems(), key=operator.itemgetter(1))
>>> print sorted_x
[('a', 1), ('c', 2), ('b', 3)]
>>> sorted_x = sorted(x.iteritems(), key=operator.itemgetter(1), reverse=True)
>>> print sorted_x
[('b', 3), ('c', 2), ('a', 1)]

Ref: http://stackoverflow.com/questions/613183/python-sort-a-dictionary-by-value

from collections import defaultdict
 
one_dimensional = defaultdict()
two_dimensional = defaultdict(defaultdict)
three_dimensional = defaultdict(lambda: defaultdict(defaultdict)
three_dimensional_with_default_int = defaultdict(lambda: defaultdict(lambda: defaultdict(int)))

Ref: http://slacy.com/blog/2010/05/python-multi-dimensional-dicts-using-defaultdict/

Sometimes when I tried to backup mysql I will encounter this problem:

mysqldump: Got error: 1016: Can't open file: './db_name/xxx_table.frm' (errno: 24) when using LOCK TABLES

This message will pop out if the table is too large to dump

The solution is simply add a

--lock-tables=false

to the mysqldump options.

Therefore the most used mysql backup command is as following:

mysqldump -u username -p db_name | gzip -c | cat > $(date +%Y-%m-%d-%H.%M.%S).sql.gz

So I found a .screenrc file to enable ctrl + arrow to switch them.

Here goes:

vbell off
 
term $TERM
termcapinfo $TERM "Co#256:AB=\E[48;5;%dm:AF=\E[38;5;%dm"
 
# saved as ~/.screenrc
caption always "%{= bK} %{= bG} [%n]%t @ %H %{-} %= %{= bR} %l %{-} | %{= bG} %Y-%m-%d %{-}"
hardstatus alwayslastline " %-Lw%{= BW}%n%f %t%{-}%+Lw %=| %0c:%s "
 
# Ctrl + left : last page
# Ctrl + right : next page
bindkey "^[[1;5C" next
bindkey "^[O5C" next
bindkey "^[[C" next
bindkey "^[[1;5D" prev
bindkey "^[O5D" prev
bindkey "^[[D" prev
 
bindkey "^[O3D" prev
bindkey "^[O3C" next
 
# C-a b --(switch to)--> view big5 data
bind b encoding big5 utf8
# C-a u --(switch to)--> view utf8 data
bind u encoding utf8 utf8

To save the time I write a very very simple shell script to run it parallelly.

To prevent overloading for server, I sleep 300 seconds after execute 5 process parallelly.

#!/bin/bash
for dataset in `seq 1 10`;do
    for prob_type in 'us' 'merge';do
        for smooth in '0.0' '0.5' '1.0' '1.5' '2.0';do
            ./my_prog.py $dataset $smooth $prob_type &
        done
        sleep 300
    done
done

I always like the color theme in ubuntu.

Yesterday I knew the color theme is called tango

so I change the iTerm color theme to be looked like tango style

Ref: Tango color theme:

http://tango.freedesktop.org/Tango_Icon_Theme_Guidelines

Just open Bookmarks -> Add new Display Profiles and called it "tango"

Then change the color in the right side to tango color theme

Then you can get the tango style theme in iTerm!

So this article is about how to push a running process in ubuntu into background.

1. Pause the running process

press ctrl+z

2. Find out the process id

$> jobs

3. restart the process in background (assuming id = 1)

$> bg 1

4. escape the process from the current shell

$> disown -a

That’s all