how to send email from terminal in ubuntu

Follow this article to setup.
while you should use application password as your password in ~/.msmtprc.
You can generate application password from here .

Use Java and Python to write a small tool

 I wrote a tool to parse fail result files of some integration tests. I wrote it in Java and in Python, it turns out using Python is far more easy to write and to read. Bellowing are some comparisons.

 What I want to do:

1. Read all txt files in a given folder.
2. Check if every txt files contains a specific string
3. Parse some information from those files which doesn't contains the specific string.

=========================================================

 Firstly, find all *.txt in a given folder.

• In Java 7: 

      final Pattern p = Pattern.compile("(.*).txt");
      File[] files = this.listFiles(new FileFilter() {
        @Override      public boolean accept(File file) {
          return p.matcher(file.getName()).matches();
        }
      });
  

  
  
• In Python 2: 
  

  files = [ f for f in os.listdir(folder_path) if re.match('(.*).txt',f)]

My comments: There are so many verbose code to create a filter to filter file name in Java 7. All I want to do is to type *.txt somewhere and just give me all txt files in the folder. Every code other than the filter (*.txt) are redundant except specifying the folder. 

=========================================================         

Next, check if a file contains the string "Failures: 0, Errors: 0, Skipped: 0"

• In Java 7:  I copy some codes from http://stackoverflow.com/questions/5600422/method-to-find-string-inside-of-the-text-file-then-getting-the-following-lines, so my method looks like this:
  

  public boolean containsText(String keyword){
    File file = new File(this.getPath());
    try {
      Scanner scanner = new Scanner(file);
      //now read the file line by line...
      while (scanner.hasNextLine()) {
        String line = scanner.nextLine();
        if(line.contains(keyword)) {
          return true;
        }
      }
    } catch(FileNotFoundException e) {
      //handle this  }
    return false;
  }

• In Python 2:  I copy codes from http://stackoverflow.com/questions/4940032/search-for-string-in-txt-file-python, so my code looks like this:    

  if not 'Failures: 0, Errors: 0, Skipped: 0' in open(folder_path+'/'+file).read():
       // do something

My comments: To check if a file contains a specific string, either we check it line by line or we read all text in the file into a string and check if the string contains the target. I really don't care which way I use since the file size is not too big. Python just tells what I want to do in one line. Actually I hope I can type "if not "xxx" in File(path)" to make my code more describable. 

=========================================================
 Last step, for each file which doesn't contains "Failures: 0, Errors: 0, Skipped: 0", extract all fail test names. Lines contains a fail test name looks like: "testSetStationOnOff(com.live365.api.station.integration.StationTestIT)  Time elapsed: 0.066 sec  <<< FAILURE!", so the format is "[test name]([test class name]) Time elapsed: [seconds] sec <<< FAILURE!", and I just want to extract the [test name] part.

• In Java 7:     

  //print fail test names  Scanner scanner = new Scanner(report);
  while (scanner.hasNextLine()) {
    String line = scanner.nextLine();

    Pattern pattern = Pattern.compile("(.*)\\((.*)\\)  Time elapsed: (.*) sec  <<< FAILURE!");
    if(pattern.matcher(line).matches()){
      Matcher matcher = pattern.matcher(line);
      matcher.find();
      println("\t"+matcher.group(1));
    }
  }

• In Python 2:

  #print fail test nameslines = open(report).read();
  results = re.findall('(.*)\((.*)\)  Time elapsed: (.*) sec  <<< FAILURE!',lines)
  for result in results:
      print '\t'+result[0]

My comments:  Yes, what I want to do is to find all strings matches the format! Again in Python my purpose is so obviously shown in two line, but in Java I have to write so many details which I really don't care about. For example, Pattern.compile(...). Why can't I just type line.matches("xxx") and Java handles the rest for me?

Timeout and retry in shell script

最近工作上需要寫一個script每隔10分鐘就重複做A,B,C三件事情,但是有一個問題,A在正常情況下30秒內做完了,但是一旦出問題, 就會一直停在那邊,所以常常等我心血來潮來看看它跑得怎麼樣了的時候,就發現他已經卡在A步驟好幾個小時了. 所以我需要Timeout function! 而且最好再來個retry機制.

timeout很簡單,只要搜尋 "linux timeout" , "linux command timeout" ...等等,馬上就會用了.

timeout 2s ping 127.0.0.1

那retry呢? retry就用一個for loop,重複做n次,如果n次以內有執行成功就break,否則就做到n次為止, 完整的 shell script範例如下:

========================================

#! /bin/bash

RETRY_LIMIT=3
TIME_OUT=3
count=0
for (( n=0; n<$RETRY_LIMIT; n++));
do
  count=$(($count+1))
  printf "run do_something $count times\n\n"
  timeout $TIME_OUT ./do_something.sh
  status=$?
  printf "\n"
  if [ $status -eq 0 ]; then
    echo "break for loop"
    exit 0
  fi
done
exit 1

=======================================

ps. do_something.sh 是一個為了測試這個timeout and retry寫的script,

他會隨機的sleep 1到6秒,script如下:

=======================================

#!/bin/bash
echo "start running do_something()"
num=$[$RANDOM%6+1]
echo "sleep $num seconds"
sleep $num
echo "end of do_something()"

=======================================

自製Arduino

 之前做了一個舉牌機器人的project, 使用我的Arduino去控制伺服馬達. 當我做完後,面臨到一個問題: Arduino很貴啊,我不把它拆下來,就不能拿它繼續玩其他project了,但是我不希望拆掉我的舉牌機器人啊. 於是我用之前研究的自製arduino來代替我買的Arduino,如下圖.

具體作法可以參考

在麵包板上建構Arduino（ATmega328P加USB轉TTL Serial轉接卡）

成本總共大約 200左右:

麵包板 68元 (在良興買的)
ATMEGA328P-PU 52元  (在露天拍賣買的)
按鍵開關 1.5*2=3元 (在露天拍賣買的)
usb to uart 轉板 68元 (在露天拍賣買的)
石英振盪器 3元 (在露天拍賣買的)
電阻電容  沒算,一定小於10元 (在台北車站地下街的電子材料行買的)

心得:

1. 最花時間的是把材料買齊
2. USB to UART 轉板價錢差異很大,因為我想知道極限在哪,所以故意買最便宜的,實際上拿來用也還沒出什麼問題
3. 線路還蠻簡單的,照著葉難網誌上的圖一下子就接好了.
4. 燒錄時在IDE上按下download以後,要趕快按麵包板上的按鈕,才會開始燒錄,不然會燒錄失敗. 燒錄失敗就再試一次就好了,多試幾次就會了. 每次要燒錄都要按按鈕有點麻煩,建議開發還是用正常的Arduino板子,等到想要把project保存下來再轉移成這種方式.

我有很多想寫的程式,大部分的idea我都真的有用程式寫過了,雖然鳥鳥的.
例如我想要寫一個flickr app,語音命令程式, 紀錄工作日誌的application.

有一個程式我構思了很久,一個可以看到自己內部結構的程式.想像程式可以
像汽車一樣打開它的引擎蓋,打開後你可以看到程式是由哪些零件組成的,
並且可以看到零件之間的相互關係.

然後你可以像改車一樣,把某個零件拔下來,換上一個新的零件,
而蓋上引擎蓋後,程式一樣可以正常運作.

這讓不懂你程式的人,可以藉由打開引擎蓋,換上他從某個管道取得的零件,
達到他要的修改.

如果是我,我就會想要像這樣把程式的引擎蓋打開,享受這種類似改機的樂趣.

[git] set up the local branch to track another remote branch

At first, I have a local code base "A", which I  git-clone it from a repo server. 

For some reasons, I would like to have another local code base "B", which contains the same things as "A".

I think it would be faster if I clones "A" to "B". 

so I do it as follow: 

git clone <path_of_A>/.git <path_of_B>

Then I got a code base B which tracks A insteads of tracking the repo sever. 

It means that, if I want to sync code base "B" with  the repo sever, I have to sync "A" with remote repo first,then sync "B" with "A". 

Since I only cares about changes in the repo server, I don't care about changes in "A". 

I removed the remote track of "A" and add a remote track of the repo server:

git remote add repo-server git@<IP_of_repo_server>:xxxx/xxx.git

git remote rm origin (origin is correspoing to "A")

Then I makes the local maser branch in "B" to track the repo server.

git branch --ser-upstream master remotes/repo-server/<the_remote_branch_name>

dump_stack()

How can it works for you

當你想知道在kernel裡面某個function是如何一層一層的被呼叫到的，你只要在該funciton裡加上一行 "dump_stack()"，

當程式跑到那一行，就會把整個code stack印出來

How to use 

以下為使用的範例，例如我想知道wifi driver裡面的wifi_set_cardetect是怎樣被呼叫的，就可以在該funciton內加一行dump_stack()。

----------------------------------------------------------

kernel/drivers/net/wireless/bcm4329/dhd_linux.c

int wifi_set_carddetect(int on)
{
        printk("%s = %d\n", __FUNCTION__, on);
dump_stack();
#ifdef CONFIG_WIFI_CONTROL_FUNC
        if (wifi_control_data && wifi_control_data->set_carddetect) {
                wifi_control_data->set_carddetect(on);
        }
#endif
        return 0;
}

----------------------------------------------------------

執行結果如下:

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

<4>[  861.317167] wifi_set_carddetect = 1
<4>[  861.320835] [<c0047a04>] (unwind_backtrace+0x0/0xf0) from [<bf003b68>] (wifi_set_carddetect+0x1c/0x54 [bcm4329])
<4>[  861.331281] [<bf003b68>] (wifi_set_carddetect+0x1c/0x54 [bcm4329]) from [<bf003c10>] (wifi_probe+0x34/0x50 [bcm4329])
<4>[  861.342126] [<bf003c10>] (wifi_probe+0x34/0x50 [bcm4329]) from [<c027ac68>] (platform_drv_probe+0x18/0x1c)
<4>[  861.351949] [<c027ac68>] (platform_drv_probe+0x18/0x1c) from [<c0279ac8>] (driver_probe_device+0xc8/0x184)
<4>[  861.361767] [<c0279ac8>] (driver_probe_device+0xc8/0x184) from [<c0279be4>] (__driver_attach+0x60/0x84)
<4>[  861.371318] [<c0279be4>] (__driver_attach+0x60/0x84) from [<c02792fc>] (bus_for_each_dev+0x48/0x84)
<4>[  861.380523] [<c02792fc>] (bus_for_each_dev+0x48/0x84) from [<c0278c50>] (bus_add_driver+0x9c/0x20c)
<4>[  861.389655] [<c0278c50>] (bus_add_driver+0x9c/0x20c) from [<c027a1dc>] (driver_register+0xa8/0x138)
<4>[  861.398851] [<c027a1dc>] (driver_register+0xa8/0x138) from [<bf03408c>] (dhd_module_init+0x8c/0x1cc [bcm4329])
<4>[  861.409092] [<bf03408c>] (dhd_module_init+0x8c/0x1cc [bcm4329]) from [<c003c588>] (do_one_initcall+0x94/0x164)
<4>[  861.419271] [<c003c588>] (do_one_initcall+0x94/0x164) from [<c00ac7d4>] (sys_init_module+0x90/0x1ac)
<4>[  861.428567] [<c00ac7d4>] (sys_init_module+0x90/0x1ac) from [<c0041ca0>] (ret_fast_syscall+0x0/0x30)

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

How dump_stack works

dump_stack定義在kernel/arch/arm/kernel/traps.c，程式流程為

dump_stack()  --> dump_backtrace() -->unwind_backtrace() --> dump_backtrace_entry() 

其中在unwind_backtrace()中會有一個while loop，裡面透過unwind_frame()靠著當下的program counter (r15) 和 link register (r14) 一層一層的往上找，每往上找一層就呼叫

dump_backtrace_entry()把目前的位置印出來，印出來的message就像這樣: 

 [<c0047a04>] (unwind_backtrace+0x0/0xf0) from [<bf003b68>] (wifi_set_carddetect+0x1c/0x54 [bcm4329])

PS: 

1. link register : 當function A的某一行呼叫funciton B時，ARM會利用這個register紀錄funciton A下一行的位址，當funciton B跑完時，CPU就知道接下來要跳回哪裡繼續執行下去。

    reference [3] [4]有ARM register的簡介。

2. unwind_frame() 裡面具體的作法可以參考[5][6] ，Stack unwinding 似乎是ARM support的一種功能，詳細請參考[6]中的pdf

Reference 

[1] 簡單實現 dump_stack

[2] Enabling Stack Dumping in Linux Kernel

[3] ARM Register introduction 

[4] armregister.jpg

[5] unwind.c in kernel 

[6] Exception Handling ABI for the ARM Architecture