When I’m developing GooStats, I encountered the problem that I need to link against an external library GooFit which contains GPU code. If I link against GooFit library directly, the linker will give strange compilation errors, and here I post how I solve it as well as the principle of cuda linking models.

 

A short summary of this article is

  1. You need to do an extra step to link all GPU objects into one library using nvcc –device-link or cmake CUDA_LINK_SEPARABLE_COMPILATION_OBJECTS macro
  2. After that, you need to add this library as the last library to your linker when creating executables or libraries.

CUDA compilation model / linking model

For normal cxx code, you always compile source file into object files first, then link object files and/or libraries into executables or libraries. You can compile source code when the implementation of some functions are missing, however they must be found during the linking phase.

Cuda code contains both device (GPU) code and hosts (CPU) code and can only be compiled with nvcc compiler (or you can compile the CPU part only with a lot of #if __CUDACC__ directives and #include <cuda_runtime.h> which define __host__ __device__ etc. to nothing).

nvcc –device-c -o lib1.o lib1.cu

nvcc –device-c -o lib2.o lib2.cu

Each output file contains two part, the host objects and device objects and now you can link the CPU part to create static library (let’s skip dynamic library first..)

ar qc lib.a lib1.o lib2.o
ranlib lib.a

And you can “almost” use this library by linking lib.a

nvcc –device-c -o main.o main.cu

g++ -o main main.o lib.a -L/cuda/installation/path -lcudart -lcudadevrt

However it only “almost” works: the linker will complain about undefined reference to some cudaRegisterAll functions. Why “almost”? Because the GPU objects is not recognizable for hosts linker g++ and you need to wrap them to g++ recognizable objects and then link them to main.

To wrap GPU objects to g++ recognizable objects, run

nvcc –device-link -o lib_all_gpu.o lib1.o lib2.o main.o

Here is the most annoying part: you can only create one single file when wrapping all GPU objects, or you can create more, but each are independent, so you can

nvcc –device-link -o lib_all_lib.o lib1.o lib2.o

nvcc –device-link -o lib_all_main.o main.o

if all device functions in main never reference any device function/global variable/array defined in lib1 and lib2, and lib1 or lib2 never reference any device function/global variable/array defined in main.

That is, if you write your pacakge lib containing lib1.cu and lib2.cu, and link the gpu code prior

nvcc –device-link -o lib_all_gpu_in_lib.o lib1.o lib2.o

ar qc lib_with_gpu.a lib1.o lib2.o lib_all_gpu_in_lib.o

ranlib lib_with_gpu.a

and hope to use it by

nvcc –device-link -o lib_all_gpu_in_main.o main.o

g++ -o main lib_all_gpu_in_main.o lib.a main.o

you won’t be able to link lib_all_gpu_in_main.o and receive an error about undefined reference to functions defined in lib.a(in lib_all_gpu_in_lib.o )

So if you write your own libraries, you must install lib1.o and lib2.o to the installation path so that cuda can link them. 

Now you can link and create your final executable

g++ -o main lib_all_gpu.o lib.a main.o -L/cuda/installation/path -lcudart -lcudadevrt

There is another annoying thing: if you put any of lib.a and main.o after lib_all_gpu.o, you will get undefined reference to some “fatbin … ” functions in lib_all_gpu.o. If you use nm command you will see that

> nm lib_all_gpu.o | grep fatbin

             U …fatbin…

here U means this function is not defined. It is defined in lib.a:

> nm lib.a | grep fatbin

 00000 R …fatbin…

here R means defined in read-only section So it is defined, we just need to switch the order.. If you would like to understand it fully, you should read the magic of static linking

You can find the full example code on my github repository. if you want to compile it with cmake, you need to download the whole sandbox repository, because CUDAsafeAddLibraryAndExe.cmake is in sandbox/cmake folder.

reference: Nvidia CUDA compilation documents

For the first time, I try to collaborate with my colleagues on git. Because we are afraid of ruining the master branch, I add the pre-receive hook on the remote server. At first I was trying to use the /dev/tty as suggested in

https://dev.ghost.org/prevent-master-push/

, however I realize for the remote server the ssh does not provide the prompt. I decide that the master can only be upated if the change has been submitted to stable_release branch.

2017.03.10 update:
(1) deleting stable_release and master branch is not allowed.
(2) new branch can be added only if it ends with _dev.

The code is the following:

# The "post-receive" script is run after receive-pack has accepted a pack
# and the repository has been updated.  It is passed arguments in through
# stdin in the form
#  &lt;oldrev&gt; &lt;newrev&gt; &lt;refname&gt;
# For example:
#  aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
#
# see contrib/hooks/ for a sample, or uncomment the next line and
# rename the file to "post-receive".

#. /usr/share/git-core/contrib/hooks/post-receive-email
while read oldrev newrev refname
do
  # not push
  if [ "<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-132a9baa84fad69ac3815df24d0857c2_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#123;&#111;&#108;&#100;&#114;&#101;&#118;&#58;&#40;&#45;&#52;&#41;&#125;&#34;&#32;&#61;&#61;&#32;&#48;&#48;&#48;&#48;&#32;&#93;&#59;&#32;&#116;&#104;&#101;&#110; &#32;&#32;&#32;&#32;&#105;&#102;&#32;&#91;&#32;&#34;&#95;&#100;&#101;&#118;&#34;&#32;&#33;&#61;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="19" width="331" style="vertical-align: -5px;"/>{refname:(-4)}" ]; then
      echo "only add new username_dev branch is allowed"
      exit 1
    fi
  fi
  if [ "<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-3e22c1b1818bf1acc68cdf4f742fd3d1_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#123;&#111;&#108;&#100;&#114;&#101;&#118;&#58;&#40;&#45;&#52;&#41;&#125;&#34;&#32;&#33;&#61;&#32;&#48;&#48;&#48;&#48;&#32;&#93;&#59;&#32;&#116;&#104;&#101;&#110; &#32;&#32;&#32;&#32;&#98;&#114;&#97;&#110;&#99;&#104;&#61;" title="Rendered by QuickLaTeX.com" height="19" width="294" style="vertical-align: -5px;"/>(git rev-parse --symbolic --abbrev-ref <img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-4cd4be41074d966df2fa31e6873b0e78_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#114;&#101;&#102;&#110;&#97;&#109;&#101;&#41; &#32;&#32;&#32;&#32;&#105;&#102;&#32;&#91;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="19" width="107" style="vertical-align: -5px;"/>{newrev:(-4)}" == 0000 ]; then
      if [ "master" == "<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-fc7832d73c2ac5666786f7add88bb15c_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#98;&#114;&#97;&#110;&#99;&#104;&#34;&#32;&#93;&#32;&#124;&#124;&#32;&#91;&#32;&#34;&#115;&#116;&#97;&#98;&#108;&#101;&#95;&#114;&#101;&#108;&#101;&#97;&#115;&#101;&#34;&#32;&#61;&#61;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="19" width="246" style="vertical-align: -5px;"/>branch" ]; then
        echo "deleting master branch and stable_release branch is not allowed."
        exit 1
      fi
    fi
    if [ "master" == "<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-fc7832d73c2ac5666786f7add88bb15c_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#98;&#114;&#97;&#110;&#99;&#104;&#34;&#32;&#93;&#32;&#124;&#124;&#32;&#91;&#32;&#34;&#115;&#116;&#97;&#98;&#108;&#101;&#95;&#114;&#101;&#108;&#101;&#97;&#115;&#101;&#34;&#32;&#61;&#61;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="19" width="246" style="vertical-align: -5px;"/>branch" ]; then
      stable_head="<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-c2492814b8eb878897d6fa341adfff40_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#40;&#103;&#105;&#116;&#32;&#115;&#104;&#111;&#119;&#45;&#114;&#101;&#102;&#32;&#45;&#115;&#32;&#115;&#116;&#97;&#98;&#108;&#101;&#95;&#114;&#101;&#108;&#101;&#97;&#115;&#101;&#41;&#34; &#32;&#32;&#32;&#32;&#32;&#32;&#105;&#102;&#32;&#91;&#32;&#34;&#109;&#97;&#115;&#116;&#101;&#114;&#34;&#32;&#61;&#61;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="19" width="404" style="vertical-align: -5px;"/>branch" ]; then
        if [ "<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-b5935252b939963dd02df6277f399b1a_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#110;&#101;&#119;&#114;&#101;&#118;&#34;&#32;&#33;&#61;&#32;&#34;" title="Rendered by QuickLaTeX.com" height="12" width="102" style="vertical-align: 0px;"/>stable_head" ]; then
          echo "The stable_release is not updated. push to stable release first"
          echo "If you are a normal user, don't try to push to master branch. Please work your own dev branch. You can push your dev branch. Follow these instructions(change username to your own):"
          echo "git checkout -b username_dev"
          echo "git branch -D master"
          echo "git checkout master"
          echo "git checkout username_dev"
          echo "git push origin username_dev"
          exit 1
        else
          echo "You are pushing to the master branch. Be careful and be responsible for what you did. Now I give you 7 seconds to regret"
          for((i=0;i&lt;7;++i)); do
            printf "."
            sleep 1
          done
          echo ""
          exit 0
        fi
      fi
      exit 0
    fi
  fi
  exit 0
done

There are two ways to set the mod rewrite: in the .conf file or in the .htaccess file.

To make the change in the .conf in effect, you need to service httpd restart. The change in .htaccess will take effects immediately. Besides, .htaccess only affects the requests in that folder.

The options that can be modified by .htaccess is specified by the AllowOverride option. It can only be put inside the <Directory> section. For Apache before 2.3.9 the default value is All, and since that it is None. The official document check here:

    https://httpd.apache.org/docs/2.4/mod/core.html#allowoverride

To make my rewrite rules effective in the whole site, I put them in the .conf file.

(1) Direct the subdomain to https + sub folder
(2) If there is no subdomain, direct it to the https

  RewriteEngine On
  RewriteEngine On
  RewriteCond %{HTTP_HOST} ^(.*)\.(.*\..*)<img src="https://dingxf.cn/blog/wp-content/ql-cache/quicklatex.com-4129c0cc539b93a321a3052399fd9525_l3.png" class="ql-img-inline-formula quicklatex-auto-format" alt="&#91;&#78;&#67;&#93; &#32;&#32;&#82;&#101;&#119;&#114;&#105;&#116;&#101;&#82;&#117;&#108;&#101;&#32;&#94;&#40;&#46;&#42;&#41;" title="Rendered by QuickLaTeX.com" height="22" width="166" style="vertical-align: -5px;"/> https://%2/%1%{REQUEST_URI} [L,R=301]
  RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

That’s all. Enjoy.

Key words:
rewrite subdomain to sub folder
redirect http to https
rewrite http to https

In some institute part of the salary is delivered as lunch coupons. These coupons could be used in limited number of shops. Our target is to make a google my map based on information from repas.it.

Here I provide a half-automatic way.

  1. Go to repas.it and choose “search”
  2. choose the city and click on “search” button, and wait until the list of shops appeared.
  3. save the page as a html file, or so-called page source.
    • for example. here you can download the page source for L’Aquila:
  4. You can check its content with vi or emacs. The first two numbers are gps co-ordinate. They are quite imprecise.. repas people are quite lazy.
  5. use my script to grab the gps co-ordinate, the address and the name of the shop from this file. You will get a file called output.csv.
    #!/bin/python
    #coding=utf-8
    import HTMLParser
    
    html_parser = HTMLParser.HTMLParser()
    
    rfile='Cerca il locale.htm'
    
    with open(rfile) as f:
        content = f.readlines()
        results=[]
        for line in content:
            result=[]
            if not line.strip().startswith('document.locations.push'):
                continue
            processed=line.strip("\t\t\tdocument.locations.push([").strip("]);\n").split("', ")
            Ngps=(processed[0].strip("'"))
            Egps=(processed[1].strip("'"))
            result.append(Ngps)
            result.append(Egps)
            p2=html_parser.unescape(processed[2].strip("'").strip('"')).split("&lt;br /&gt;")
            for p3 in p2:
                p3pos=-1
                while not p3.find("&lt;",p3pos+1)==-1:
                    p3pos=p3.find("&lt;",p3pos+1)
                    p3rpos=p3.find("&gt;",p3pos+1)+1
                    p3=p3[0:p3pos]+p3[p3rpos:]
                    p3pos=-1
                p4=p3.replace(","," ").replace("\\","")
                result.append(p4)
            results.append(result)
    
    wfile='output.csv'
    wf=open(wfile,"w")
    wf.write("GPS1,GPS2,Name,Address\n")
    for result in results:
        wf.write(result[0]+","+result[1]+","+result[2]+","+result[3]+"\n")
    wf.close()
    

– You can try to download
process.py
– and rename it as “process.py”

  1. open google mymap, create a new map, and import the csv file we just create. I think you should know what to do next.

Good luck

.

When charged particle deposited energy in the liquid scintillator, the energy will be transferred to the primary fluor and emits scintillation light. However they are not proportional. The problem is the birks law.

Today I found an article and they write an rountine to calculate it.. very nice!

And it can be downloaded from  http://www.cpc.cs.qub.ac.uk

Good place.

Besides, I will check if I can find a way to write article with MarkDown, and upload picture easily.

Now it seems work!

#include <iostream>;
int main(int argc, char *argv[]) {
    return 0;
}

test:

(1)   \begin{equation*} \dfrac{\beta}{\alpha} = \Gamma(\beta-\alpha) \end{equation*}

[
\sin 2
]
5+3\times 7

(2)   \begin{equation*} \dfrac{\beta}{\alpha} = \Gamma(\beta-\alpha) \end{equation*}

[
\sin 2
]
5+3\times 7

服务器的内存爆了,swap都用爆了。一开始想着linux本来就会把内存吃光,但一想把swap吃光就不科学了。研究了半天,用netstat发现有一对imap连接。。原来是我的foxmail的问题。

把foxmail关了就没事了。ram使用量、连接数、网络io,disk io立马全都下来了。

坑爹的foxmail!!

 

9月23日更新:

查了一下,内存依然几乎用爆,用top看了一下,是httpd用爆的。httpd -l看了是工作在prefork模式下,遂修改参数,内存使用量立马下来了。参数没有完全理解,只是望文生义。

<IfModule prefork.c>
StartServers 2
MinSpareServers 3
MaxSpareServers 5
ServerLimit 64
MaxClients 64
MaxRequestsPerChild 200
</IfModule>

倒腾了一天,想把cocos2d-x撞到centos 6上。各种库版本太低,首当其冲就是gcc:gcc4.4.7不支持c++11。为了装gcc,我的u盘爆了(gcc要4个G),于是重新分了个盘,改了fstab挂成了home盘。还有各种版本问题。。

不想写了,一把辛酸泪 T T

装CentOS 7都好些,Fedora也不错

昨天没有更新文章,因为这两天都在忙着开发一个德州扑克的游戏。项目是公开的,挂在github上:https://github.com/DingXuefeng/Nayav 本来是想用object C来写的,后来还是决定用我最熟悉的C++来写了。

这个项目的终极目的是编写一个可以和人类抗衡的扑克AI,有自己的个性,会使诈,bluff the bluffer等等,并且极度冷静。如果能有别人一起开发就最好啦。

目前我有如下几个体验:

  1. 实现一个看上去简单(但往往超级复杂)的功能,一定要一步一步,先写一些dummy method。
  2. 如果一个class太大了,应该refraction把它分成两个或者更多的小类。重构的技巧是把所有的m_xx的类变量变成Get_xx()的方法,然后把Get_xx()内容从 return m_xx换成return Get_secondClass()->Get_xx();,然后就可以把m_xx变量直接搬到secondClass中去了。可以写两个get,分别对应Gettter和Setter:
    1. const X& Get_xx() const { return xx; }
    2. X& Get_xx() { return xx; }
  3. 如果一个method里面不包含本类的任何属性,这个method可以安全的移动到secondClass。

一个例子

class A {

public:

A : m_a(0) {};

void initialize_a() { m_a= 99; };

void doSomething() { m_a += 1; return m_a; };

private:

int m_a;

};

首先变成这样:

class A {

public:

A : m_a(0) {};

void initialize_a() { Get_a() = 99; };

void doSomething() { Get_a() += 1; };

private:

const int &Get_a() const { return m_a; }

int &Get_a() { return m_a; }

int m_a;

};

然后

class A {

public:

A : m_a(0) {};

void initialize_a() { Get_a() = 99; };

void doSomething() { Get_a() += 1; };

private:

const int &Get_a() const { return Get_b()->Get_a(); }

int &Get_a() { return Get_b()->Get_a(); }

B m_b;

};

class B {

private:

const int &Get_a() const { return m_a; }

int &Get_a() { return m_a; }

int m_a;

friend class A;

};

最后把method也搬过来

class A {

public:

void initialize_a() { Get_b()->inialize_a(); };

void doSomething() { Get_b()->doSomething(); };

private:

B m_b;

};

class B {

public:

B() : m_a(0) {};

void initialize_a() { Get_a() = 99; };

void doSomething() { Get_a() += 1; };

 

private:

const int &Get_a() const { return m_a; }

int &Get_a() { return m_a; }

int m_a;

friend class A;

};

 

 

 

 

今天好累呢,就少写一点吧。

上网其实就是向远程服务器下载网页,浏览器通过翻译把网页变成我们现在看到的样子。我们在浏览器地址栏输入www.gmail.com到最后我们看到gmail登陆界面分成两个过程:

  1. 请问“www.gmail.com”的IP地址是多少?dns服务器会告诉你,“www.gmail.com的ip地址是216.58.221.133”。
  2. 请转交给216.58.221.133,内容为【我想下载“/ ”这个网页】。

什么叫中间人攻击呢?

假设你在Starbuck上wifi,想登陆gmail邮箱,那么你就会问www.gmail.com的ip地址是多少。正常情况下你应该等你的dns服务器告诉你,但是某个黑客会告诉你:www.gmail.com的ip地址是10.0.1.99(其实是黑客电脑的ip地址)。

你会问:我有那么傻么,为啥我不看看这个是不是8.8.8.8发来的?你应该继续想想,你该看什么?有什么是不能伪造的?事实上,你询问www.gmail.com的那条消息会被拦截,最后你的dns根本收不到你的请求,你只会收到一份应答,你不会怀疑它的真实性。

接下来你在浏览器里面输入www.gmail.com之后,访问的其实是黑客提供给你的一个假网站(跟真的一模一样)。你输入的密码会被黑客记录下来。然后黑客会把你的请求转发给gmail,gmail以为是黑客在登陆你的邮箱,而且账号密码是对的,gmail会把你邮箱的信件发送给黑客,黑客再把内容转交给你。

这有点像中学时候传纸条,男生A坐在最后一排,女生B坐在第一排,男生A给女生B传纸条,需要经过也喜欢女生B的男生C!

男生C会偷看纸条的内容(窃取密码等)。如果男生C想要改写纸条的内容,他需要把男生A的每张纸条都撕掉,然后自己写一份纸条,这样自己的纸条前后字迹看起来才是一样的,女生B也不会怀疑,为什么两封信的字迹不一样。

那么有个问题,如果女生B跑去问老师,这个纸条的字是谁写的,这不就露馅了么?这个想法很好,这是SSL根证书策略的精华:

  1. 根据字迹判断身份(https协议下ssl证书保证网页内容的发布方身份)
  2. 由老师的信誉保证(1)的判断是真实的(ssl证书中会说明其颁发单位,该颁发单位保证证书是真实有效的)。

所以,https协议相对更加安全。