ArchLinux 部署 SSD 缓存

在 Linux 系统上使用带宽更高、延迟更小的 SSD 作为 HDD 的缓存来打造软件实现的“混合硬盘”是一种容量和性能折中的方案。在 Linux 系统中使用高速外存作低速外存的缓存有两个成熟的方案:1. lvmcache 2. bcache。本文记录的是基于 lvmcache 在 Arch Linux 系统上的部署方法。

硬件情况
1. 一块120GB容量的固态硬盘 (/dev/sda)。
2. 一块1TB容量的机械硬盘 (/dev/sdb)。

分区规划

/dev/sda1     20GB     lvm
/dev/sda2     100GB    lvm
/dev/sdb      1TB      lvm

LVM 规划

VG (system) -> { PV (/dev/sda1) }
VG (data) -> { PV (/dev/sdb), PV (/dev/sda2) }
 
LV (system/arch) 19.9GB
LV (data/home) 917GB
LV (data/home_cache) 91GB

操作系统完全安装在 SSD 中, home 存放于 HDD 中,但使用 SSD 的剩余空间作为 cache。为什么不把所有数据存放于 HDD 中,仅用 SSD 作为 cache 呢?测试结果是当 cache 加入后就影响了系统的启动,所有只能应用于数据区。

详细步骤
1. 使用 ArchLinux iso 启动盘启动系统。
2. fdisk /dev/sda 先创建一个类型为 lvm 容量为 20GB 的标准主分区,再创建一个类型为 lvm 容量为 100GB 的标准主分区。
3. pvcreate /dev/sda1
4. pvcreate /dev/sda2
5. pvcreate /dev/sdb
6. vgcreate system /dev/sda1
7. vgcreate data /dev/sdb
8. lvcreate -L 19.9G system -n arch
9. mkfs.ext4 -E discard /dev/mapper/system-arch
10. mount /dev/mapper/system-arch /mnt
11. pacstrap /mnt 按照需要安装系统
12. arch-chroot /mnt
13. lvcreate –type cache –cachemode wirteback -L 91G -n home_cache data/home /dev/sda2
14. 另开一个 tty, mount -o bind /run /mnt/run
15. 回到原 tty, vim /etc/mkinitcpio.conf,找到 HOOkS,在 block 和 filesystem 之间增加个 lvm2
16. mkinitcpio -p linux
17. vim /etc/default/grub,找到 GRUB_PRELOAD_MODULES,增加 lvm。
18. grub-mkconfig -o /boot/grub/grub.cfg
19. grub-install /dev/sda
20. 退出,重启。

查看 cache 状态

sudo lvs -o cache_read_hits,cache_read_misses,cache_write_hits,cache_write_misses data/home

Over!

优化 Linux 系统 IBus 郑码输入法

偶然的机会体验了一下 Windows 平台的“东方制作”郑码6.6,默认为4键自动提交,不动态调频等等觉得很好用,用了这么长时间的 IBus 郑码都不知道这些都应该是形码输入法的福利了。查找下来发现 IBus 郑码不好用的原因其实主要为配置不当和对5码郑码编码方案支持的不成熟。

在原来 ibus-table-zhengma 的基础上优化了郑码的使用体验。主要包括启用自动提交、关闭自动调词频,另外还将5码编码方案退回4码编码方案。

码表下载
https://github.com/heiher/ibus-table-zhengma

安装、配置
ArchLinux 安装方法:
1. 从 AUR 下载源代码包 https://aur.archlinux.org/packages/ibus-table-zhengma
2. 通过 makepkg 命令编译二进制包并安装。

配置注意事项:
如原来使用过旧版的郑码输入法,请在安装新版本后,删除 ~/.local/share/ibus-table 目录,并执行 ibus restart,最后进入郑码输入法的配置界面恢复默认配置(否则自动提交功能不会正常工作)。

Over!

Linux 不使用 chroot 临时替换C运行时库

有些时候我们需要在老旧的 Linux 系统上运行一些依赖于较新版本 C 库的应用程序或库,应用程序会因为系统中安装的C库缺少符号还启动失败。解决方法之一就是临时替换使用非系统安装的C运行时库。使用临时C库需要做些什么配置及会带来哪些问题呢?

配置步骤
1. 下载与目标应用程序版本相匹配的临时C库,解压缩到临时位置 A。
2. 需要设置 LD_LIBRARY_PATH 环境变量指向目标临时C库的存储位置 A。
3. 需要通过与临时C库匹配的 ld.so 启用应用程序。因为应用程序默认是链接了一个绝对路径的 ld.so,如 x86_64 是 /lib64/ld-linux-x86-64.so.2

衍生问题
使用临时C库的 ld.so 启动的应用程序执行系统标准命令的子进程出错,原因是因为环境变量 LD_LIBRARY_PATH 被子进程继承,从而导致子进程在执行系统C库的ld.so中加载了版本不匹配的临时C库。

解决办法
在合适的时机清除环境变量 LD_LIBRARY_PATH,最合适的时机应用就是执行目标应用程序 main 函数之前啦。这里又要用到了之前写过的方法 => Linux 平台一种进程代码注入方法

/* fakemain.c
 * Heiher <admin@heiher.info>
 */
 
#include <stdio.h>
#include <stdlib.h>
 
#define __USE_GNU
#include <dlfcn.h>
 
int
__libc_start_main(int (*main)(int, char **, char **),
			int argc, char **ubp_av, void (*init)(void),
			void (*fini)(void), void (*rtld_fini)(void),
			void (*stack_end))
{
	int (*__libc_start_main_real)(int (*main) (int, char **, char **),
				int argc, char **ubp_av, void (*init)(void),
				void (*fini)(void), void (*rtld_fini)(void),
				void (*stack_end));
 
	unsetenv ("LD_PRELOAD");
	unsetenv ("LD_LIBRARY_PATH");
 
	__libc_start_main_real = dlsym(RTLD_NEXT, "__libc_start_main");
 
	return __libc_start_main_real(main, argc, ubp_av, init, fini,
				rtld_fini, stack_end);
}
gcc -fPIC -O3 -shared -o libfakemain.so fakemain.c -ldl

设置环境变量 LD_PRELOAD=/xxx/libfakemain.so,运行目标应用程序在执行 main 之前即会清除 LD_PRELOAD 和 LD_LIBRARY_PATH 变量。

为了方便使用我还写了个 wrapper,使用方法是将真实的目标应用程序 xxx 重命令为 xxx.bin,然后创建个符号链接 xxx 指向 wrapper,执行时直接执行 xxx,wrapper 会自动设置所需要的环境变量。

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
 
int
main (int argc, char *argv[])
{
	int i;
	char buf[1024], path[1024];
	char *str, *root, *args[512];
 
	/* get FAKE_ROOT */
	root = getenv ("FAKE_ROOT");
	if (!root) {
		fprintf (stderr, "Please set environment FAKE_ROOT!\n");
		return -1;
	}
 
	/* export PATH */
	str = getenv ("PATH");
	if (!str) {
		fprintf (stderr, "Get environment PATH failed!\n");
		return -2;
	}
	if (NULL == strstr (str, root)) {
		strcpy (buf, root);
		strcat (buf, "/bin:");
		strcat (buf, str);
		if (0 != setenv ("PATH", buf, 1)) {
			fprintf (stderr, "Set environment PATH failed!\n");
			return -3;
		}
	}
 
	/* export LD_PRELOAD */
	strcpy (buf, root);
	strcat (buf, "/lib64/libfakemain.so");
	if (0 != setenv ("LD_PRELOAD", buf, 1)) {
		fprintf (stderr, "Set environment LD_PRELOAD failed!\n");
		return -4;
	}
 
	/* export LD_LIBRARY_PATH */
	strcpy (buf, root);
	strcat (buf, "/lib64");
	if (0 != setenv ("LD_LIBRARY_PATH", buf, 1)) {
		fprintf (stderr, "Set environment LD_LIBRARY_PATH failed!\n");
		return -5;
	}
 
	/* set new path */
	strcpy (path, root);
	strcat (path, "/lib64/ld-2.20.so");
	args[0] = path;
 
	/* set real program path */
	strcpy (buf, root);
	strcat (buf, "/bin/");
	strcat (buf, argv[0]);
	strcat (buf, ".bin");
	args[1] = buf;
 
	/* copy arguments */
	for (i=1; i<argc; i++)
	      args[i+1] = argv[i];
	args[i+1] = NULL;
 
	/* run real program */
	return execv (path, args);;
}
gcc -O3 -o wrapper wrapper.c

Over!

mips64el toolchain for x86_64

mips64el toolchain 是用于在 x86_64 平台交叉编译 mips64el 目标程序的工具集,该工具集分为两种大版本:odd-spreg 和 no-odd-spreg,其中龙芯仅适用 no-odd-spreg 版本。系统库包含 mips64el o32, n32 和 n64 多种版本的库,分别有依赖于 Linux 2.6 内核和 Linux 3.4 内核的两种版本。另外还有支持 x86_64 交叉编译 Mozilla JS 引擎的支持包。

下载
Source: mips64el-toolchain-2.src.tar.xz
no-odd-spreg
toolchain: mips64el-toolchain-2.x64.tar.xz
system libaries (Linux 2.6): mips64el-toolchain-linux-2.6-2.x64.tar.xz
system libaries (Linux 3.4): mips64el-toolchain-linux-3.4-2.x64.tar.xz
system libaries (Linux 3.4 MozJS): mips64el-toolchain-linux-3.4-mozjs-2.x64.tar.xz
odd-spreg
toolchain: mips64el-toolchain-2.x64.tar.xz
system libaries (Linux 2.6): mips64el-toolchain-linux-2.6-2.x64.tar.xz
system libaries (Linux 3.4): mips64el-toolchain-linux-3.4-2.x64.tar.xz

安装

sudo tar --numeric-owner -xf xxxx -C /

配置
设置环境变量

export PATH=${PATH}:/opt/mips64el-toolchain/bin

切换系统库

sudo ln -s -f linux-2.6 /opt/mips64el-toolchain/platforms/current

编译

# MIPS32 o32
mips64el-unknown-linux-gnu-gcc -march=mips32r2 -mabi=32 -o test test.c
# MIPS64 n32
mips64el-unknown-linux-gnu-gcc -march=mips64r2 -mabi=n32 -o test test.c
# MIPS64 n64
mips64el-unknown-linux-gnu-gcc -march=mips64r2 -mabi=64 -o test test.c

Over!

How to connect to a WPA/WPA2 WiFi network using Linux command line

This is a step-to-step guide for connecting to a WPA/WPA2 WiFi network via the Linux command line interface. The tools are:
1. wpa_supplicant
2. iw
3. ip
4. ping

iw is the basic tool for WiFi network-related tasks, such as finding the WiFi device name, and scanning access points. wpa_supplicant is the wireless tool for connecting to a WPA/WPA2 network. ip is used for enabling/disabling devices, and finding out general network interface information.

The steps for connecting to a WPA/WPA2 network are:

1. Find out the wireless device name.

    $ /sbin/iw dev
    phy#0
    	Interface wlan0
    		ifindex 3
    		type managed

The above output showed that the system has 1 physical WiFi card, designated as phy#0. The device name is wlan0. The type specifies the operation mode of the wireless device. managed means the device is a WiFi station or client that connects to an access point.

2. Check that the wireless device is up.

    $ ip link show wlan0
    3: wlan0: (BROADCAST,MULTICAST) mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
        link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff

Look for the word “UP” inside the brackets in the first line of the output.

In the above example, wlan0 is not UP. Execute the following command to bring it up:

    $ sudo ip link set wlan0 up  
    [sudo] password for peter:

Note: you need root privilege for the above operation.

If you run the show link command again, you can tell that wlan0 is now UP.

    $ ip link show wlan0
    3: wlan0: (NO-CARRIER,BROADCAST,MULTICAST,UP) mtu 1500 qdisc mq state DOWN mode DEFAULT qlen 1000
        link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff

3. Check the connection status.

    $ /sbin/iw wlan0 link
    Not connected.

The above output shows that you are not connected to any network.

4. Scan to find out what WiFi network(s) are detected

    $ sudo /sbin/iw wlan0 scan
    BSS 00:14:d1:9c:1f:c8 (on wlan0)
            ... sniped ...
    	freq: 2412
    	SSID: gorilla
    	RSN:	 * Version: 1
    		 * Group cipher: CCMP
    		 * Pairwise ciphers: CCMP
    		 * Authentication suites: PSK
    		 * Capabilities: (0x0000)
            ... sniped ...

The 2 important pieces of information from the above are the SSID and the security protocol (WPA/WPA2 vs WEP). The SSID from the above example is gorilla. The security protocol is RSN, also commonly referred to as WPA2. The security protocol is important because it determines what tool you use to connect to the network.

5. Connect to WPA/WPA2 WiFi network.
This is a 2 step process. First, you generate a configuration file for wpa_supplicant that contains the pre-shared key (“passphrase”) for the WiFi network.

    $ sudo -s
    [sudo] password for peter: 
    $ wpa_passphrase gorilla >> /etc/wpa_supplicant.conf 
    ...type in the passphrase and hit enter...

wpa_passphrase takes the SSID as the single argument. You must type in the passphrase for the WiFi network gorilla after you run the command. Using that information, wpa_passphrase will output the necessary configuration statements to the standard output. Those statements are appended to the wpa_supplicant configuration file located at /etc/wpa_supplicant.conf.

Note: you need root privilege to write to /etc/wpa_supplicant.conf.

    $ cat /etc/wpa_supplicant.conf 
    # reading passphrase from stdin
    network={
    	ssid="gorilla"
    	#psk="testtest"
    	psk=4dfe1c985520d26a13e932bf0acb1d4580461dd854ed79ad1a88ec221a802061
    }

The second step is to run wpa_supplicant with the new configuration file.

    $ sudo wpa_supplicant -B -D wext -i wlan0 -c /etc/wpa_supplicant.conf
    -B means run wpa_supplicant in the background.
    -D specifies the wireless driver. wext is the generic driver.
    -c specifies the path for the configuration file.

Use the iw command to verify that you are indeed connected to the SSID.

    $ /sbin/iw wlan0 link
    Connected to 00:14:d1:9c:1f:c8 (on wlan0)
    	SSID: gorilla
    	freq: 2412
    	RX: 63825 bytes (471 packets)
    	TX: 1344 bytes (12 packets)
    	signal: -27 dBm
    	tx bitrate: 6.5 MBit/s MCS 0
 
    	bss flags:	short-slot-time
    	dtim period:	0
    	beacon int:	100

6. Obtain IP address by DHCP

    $ sudo dhclient wlan0

Use the ip command to verify the IP address assigned by DHCP. The IP address is 192.168.1.113 from below.

    $ ip addr show wlan0
    3: wlan0:  mtu 1500 qdisc mq state UP qlen 1000
        link/ether 74:e5:43:a1:ce:65 brd ff:ff:ff:ff:ff:ff
        inet 192.168.1.113/24 brd 192.168.1.255 scope global wlan0
        inet6 fe80::76e5:43ff:fea1:ce65/64 scope link 
           valid_lft forever preferred_lft forever

7. Add default routing rule.
The last configuration step is to make sure that you have the proper routing rules.

    $ ip route show
    192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.113

The above routing table contains only 1 rule which redirects all traffic destined for the local subnet (192.168.1.x) to the wlan0 interface. You may want to add a default routing rule to pass all other traffic through wlan0 as well.

    $ sudo ip route add default via 192.168.1.254 dev wlan0
    $ ip route show
    default via 192.168.1.254 dev wlan0 
    192.168.1.0/24 dev wlan0  proto kernel  scope link  src 192.168.1.113

8. ping external ip address to test connectivity

    $ ping 8.8.8.8
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_req=1 ttl=48 time=135 ms
    64 bytes from 8.8.8.8: icmp_req=2 ttl=48 time=135 ms
    64 bytes from 8.8.8.8: icmp_req=3 ttl=48 time=134 ms
    ^C
    --- 8.8.8.8 ping statistics ---
    3 packets transmitted, 3 received, 0% packet loss, time 2000ms
    rtt min/avg/max/mdev = 134.575/134.972/135.241/0.414 ms

The above series of steps is a very verbose explanation of how to connect a WPA/WPA2 WiFi network. Some steps can be skipped as you connect to the same access point for a second time. For instance, you already know the WiFi device name, and the configuration file is already set up for the network. The process needs to be tailored according to your situation.

From: http://linuxcommando.blogspot.com/2013/10/how-to-connect-to-wpawpa2-wifi-network.html
Over!

How to disable auto suspend when I close laptop lid?

Edit /etc/systemd/logind.conf and make sure you have,

HandleLidSwitch=ignore

which will make it ignore the lid being closed. (You may need to also undo the other changes you’ve made).

Full details over at the archlinux Wiki.

The man page for logind.conf also has the relevant information,

   HandlePowerKey=, HandleSuspendKey=, HandleHibernateKey=,
   HandleLidSwitch=
       Controls whether logind shall handle the system power and sleep
       keys and the lid switch to trigger actions such as system power-off
       or suspend. Can be one of ignore, poweroff, reboot, halt, kexec,
       suspend, hibernate, hybrid-sleep and lock. If ignore logind will
       never handle these keys. If lock all running sessions will be
       screen locked. Otherwise the specified action will be taken in the
       respective event. Only input devices with the power-switch udev tag
       will be watched for key/lid switch events.  HandlePowerKey=
       defaults to poweroff.  HandleSuspendKey= and HandleLidSwitch=
       default to suspend.  HandleHibernateKey= defaults to hibernate.

Over!

Redirect TCP transmissions over Socks5 proxy

重定向 TCP 传输通过 Socks5 代理,此方案用于 Linux 平台。

下载、编译 HevSocks5TProxy

git clone git://github.com/heiher/hev-lib
cd hev-lib
make static
cd ..
 
git clone git://github.com/heiher/hev-socks5-tproxy
cd hev-socks5-tproxy
make

运行 HevSocks5TProxy

bin/hev-socks5-tproxy 127.0.0.1 10800 127.0.0.1 1080
# 127.0.0.1 : 本地监听地址
# 10800 : 本地监听端口
# 127.0.0.1 : Socks5 服务器地址
# 1080 : Socks5 服务器端口

本地全局启用
用于本地主机,重定向所有的 TCP 传输经过 Socks5 代理

sudo iptables -t nat -A OUTPUT -d xxx.xxx.xxx.xxx/32 -j RETURN # Bypass,远程 Socks5 服务器地址不通过代理
sudo iptables -t nat -A OUTPUT -m tcp -p tcp -j REDIRECT --to-port 10800

网关全局启用
用于网关服务器,重定向所有的 TCP 传输经过 Socks5 代理

sudo iptables -t nat -A OUTPUT -d xxx.xxx.xxx.xxx/32 -j RETURN # Bypass,远程 Socks5 服务器地址不通过代理
sudo iptables -t nat -A PREROUTING -m tcp -p tcp -j REDIRECT --to-port 10800

DNS 转发
对于 DNS 污染情况,参考 Forwarding DNS queries on TCP transport

Over!

Forwarding DNS queries on TCP transport

转发 UDP 协议的 DNS 查询至 TCP 协议传输,目前可以有效的抵御某些组织的 DNS 污染,此方案用于 Linux 平台。

下载、编译 DNS Forwarder

git clone git://github.com/heiher/hev-lib
cd hev-lib
make static
cd ..
 
git clone git://github.com/heiher/hev-dns-forwarder
cd hev-dns-forwarder
make

运行 DNS Forwarder

bin/hev-dns-forwarder 127.0.0.1 5300 8.8.8.8
# 127.0.0.1 : 本地监听地址
# 5300 : 本地监听端口
# 8.8.8.8 :  上游 DNS 服务器

本地全局启用
用于本地主机,透明转发所有的本机 DNS 查询

sudo iptables -t nat -A OUTPUT -m udp -p udp --dport 53 -j REDIRECT --to-port 5300

或直接设置本地的 DNS 服务器为 127.0.0.1,并将 Forwarder 的本地监听端口修改为 53。

网关全局启用
用于网关服务器,透明转发所有网关服务的主机的 DNS 查询

sudo iptables -t nat -A PREROUTING -m udp -p udp --dport 53 -j REDIRECT --to-port 5300

Over!

Linux netfilter REDIRECT target 构建透明代理原理

构建透明代理至于需要满足两个条件:1. 能够劫持应用程序产生的需代理的网络数据包。2. 能够获得这些数据包的原目的地址。

Linux netfilter 有一个 REDIRECT target,可用于 nat 表的 PREROUTING 和 OUTPUT 链上,其工作流程是将上述的两个链上命中的数据包的目的地址修改成(即重定向)该包进入网卡的主 IP 地址,如果是本地回环网络产生的包即重定向到 127.0.0.1。目标端口从参数指定,协议支持 tcp, udp, dccp or sctp。

从上面的情况看,Linux netfilter 的 REDIRECT target 可以实现透明代理的第一个条件,而第二个条件在强大的 Linux netfilter 子系统中也有方法:

#include <linux/netfilter_ipv4.h>
 
struct sockaddr_in orig_addr;
socklen_t orig_addr_len = sizeof (orig_addr);
if (0 == getsockopt(client_fd, SOL_IP, SO_ORIGINAL_DST, (struct sockaddr*) &orig_addr, &orig_addr_len)) {
        printf ("This is original destination address => %s:%u\n", inet_ntoa (orig_addr.sin_addr), ntohs (orig_addr.sin_port));
}

Over!

Firefox for Android 代理设置

目前发现 Android 平台的浏览器原生支持代理配置的仅 Firefox for Android(官方下载),不仅支持 HTTP、HTTPS 和 Socks4、5,还支持 PAC 脚本。

全局代理配置
在 Firefox 中访问 about:config,使用 proxy 过滤器并的找到 network.proxy.socks, network.proxy.socks_port, network.proxy.socks_remote_dns, network.proxy.type 并更改成图中配置。
Screenshot_2014-09-19-13-34-16

Screenshot_2014-09-19-13-34-28

自动代理配置
将 PAC 脚本放至于 /sdcard/proxy.pac,在 Firefox 中访问 about:config,使用 proxy 过滤器并的找到 network.proxy.autoconfig_url, network.proxy.type 并更改成图中配置。
Screenshot_2014-09-19-13-33-51

Screenshot_2014-09-19-13-42-48

Over!