Plan9 is designed as a distributed operating system. Therefore, the OS has many nice features for grid computing as shown in the paper by Mirtchovski et al.[1] and also with more detailed descriptions in the thesis by Mirtchovski[2].
Stimulated by their work, we 9fans devoted ourselves to grid computing. From that exercise, we got some fruitful results and ideas to the future. They are summarized in Bell-labs website[3]. However, it seems the activity had ended up without making nice pieces of ideas into reality.
What was wrong? Probably grid computing itself was useless for the people in 9fans. They are interested only in technical problems. It is still open problem what services are useful to the community.
Recently Big Data is talked about. By “Big Data” I mean those data that is inappropriate to be transferred across network. We should note that programs are much smaller then data. Therefore, the data should be processed at the server side by exporting programs to the server. Then we need remote login. In such case, security does matter.
The server I present here is a magical server for those who don't know Plan 9. Clients need to execute their programs in the server, however, the server does not allow any byte to be written into the server's storage. Is it possible to construct such a server that allows execution of the client programs? Yes possible!
If we have such a server, we need not to allocate user's storage space in the server, which means the time and labor will be reduced greatly. In addition, we can keep the server perfectly clean.
Ssh is a common tool today for remote execution to unix server. Instead, Plan 9 uses cpu command for remote execution. Cpu command of Plan 9 is different from ssh (or telnet) in that it is not only for command execution but also it mounts local file system to the remote side on the fly as illustrated in Fig.2. The mount point is /mnt/term in the server's namespace.
Therefore files in local side are visible under /mnt/term in remote side. This means we no longer need traditional tools such as ftp or scp to pass files from locale side to remote side, and furthermore, we need not edit files in remote side. They are editable in local side.
Grid computings today are mainly based on the softwares from Globus[4]. They need distributed accounting with distributed file system so that users are permitted to login and so that storage space is allocated to the users for locating their programs, which however needs high level collaboration. The mechanism illustrated in Fig.2 will make it needless to allocated users space, which means: grid systems may become greatly simplified.
The server I present here is designed dreaming to be a base model for future grid computing that allows freedom for clients to execute their programs, and also that needs perfect cleanness to keep the computing service. The server is constructed on those nice pieces of ideas discussed in plan9 users group[3].
[1] Andrey Mirtchovski, Rob Simmonds and Ron Minnich
Plan 9 – an Integrated Approach to Grid Computing
Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International
http://p9.nyx.link/9grid/plan9-grid.pdf (mirror)
[2] Andrey A. Mirtchovski
Grid Computing with Plan 9 – an Alternative Solution for Grid Computing
http://mirtchovski.com/p9/thesis.pdf (2005)
http:thesis.pdf (mirror)
[3] Plan 9 Wiki — 9grid
http://plan9.bell-labs.com/wiki/plan9/9grid/
http://p9.nyx.link/wiki/9grid/ (mirror)
http://9p.io/wiki/plan9/9grid/index.html (mirror)
[4] Globus
https://www.globus.org/
You need to be registered as a user of plan9.bell-labs.com1.
Execute
cpu -h grid.nyx.link -k 'dom=outside.plan9.bell-labs.com'
key dom=outside.plan9.bell-labs.com proto=p9sk1 user=XXXXX !password=YYYYY
XXXXX is your ID and YYYYY is your password.
If you succeed in login, you will see a prompt “grid%”.
Your name on the server is XXXXX@outside.plan9.bell-labs.com.
Of course the name is not registered as a user to my server.
Thereby you are attached as user none to my file system.
Ramdisk is provided to /tmp which is bound to /usr/none/tmp.
Try
ls /usr
/usr/none, /usr/arisawa and /usr/OTHERS, where /usr/OTHERS are yours that are produced bybind -a /mnt/term/usr /usr
You can browse files on the grid server using acme.
However you cannot run commands in acme windows because mount operation is disabled.
The user “XXXXX@outside.plan9.bell-labs.com” has ability to access your local files. Therefore you can edit your files using acme on grid server. (And of course also on your local machine.)
In processing data in grid server, you probably need your own programs. Except a few operations, the 9grid server allows executing your programs even if that are binary executables. Your commands are in /mnt/term/bin.
General speaking, servers can become dirty in multi-users environment. To keep the sanity of server, the 9grid server disables users from writing to permanent storage of the server. If you want to save something, you can write it to your own storage through /mnt/term/usr.
Look /usr/none/lib/profile for the settings and look /usr/arisawa/src/grid for the grid patches.
In accessing 9grid servers, we use cpu command. Then your processes on the 9grid server have ability to access your local machine. This means you have a potential security risk. It is safe to export only a portion of namespace of your file system to 9grid server.
Cpu command has -P option for this purpose. However, unfortunately, this option does not work well.
Another way is to construct minimum namespace for client side.
To do this, create the following files in somewhere, for example in /usr/none/lib.
term% pwd /usr/none/lib term% lr -l grid d-rwxrwxr-x arisawa sys 0 2015/12/24 13:29:26 grid d-rwxrwxr-x arisawa sys 0 2015/12/24 13:26:44 grid/ns d-rwxrwxr-x arisawa sys 0 2015/12/16 23:05:16 grid/ns/bin d-rwxrwxr-x arisawa sys 0 2015/12/24 12:54:23 grid/ns/dev --rw-rw-r-- arisawa sys 0 2015/12/24 12:54:11 grid/ns/dev/cons --rw-rw-r-- arisawa sys 0 2015/12/24 12:54:11 grid/ns/dev/consctl d-rwxrwxr-x arisawa sys 0 2015/12/24 12:54:23 grid/ns/dev/draw --rw-rw-r-- arisawa sys 0 2015/12/24 12:54:11 grid/ns/dev/random d-rwxrwxr-x arisawa sys 0 2015/12/25 09:13:14 grid/ns/env d-rwxrwxr-x arisawa sys 0 2015/12/24 02:07:26 grid/ns/mnt d-rwxrwxr-x arisawa sys 0 2015/12/17 05:41:17 grid/ns/mnt/factotum --rw-rw-r-- arisawa sys 0 2015/12/17 05:32:11 grid/ns/mnt/factotum/ctl --rw-rw-r-- arisawa sys 0 2015/12/17 05:41:17 grid/ns/mnt/factotum/log d-rwxrwxr-x arisawa sys 0 2015/12/24 02:07:26 grid/ns/mnt/wsys d-rwxrwxr-x arisawa sys 0 2015/12/16 23:15:39 grid/ns/net d-rwxrwxr-x arisawa sys 0 2015/12/24 12:52:41 grid/ns/proc d-rwxrwxr-x arisawa sys 0 2015/12/25 09:25:52 grid/ns/usr d-rwxrwxr-x arisawa sys 0 2015/12/25 09:25:52 grid/ns/usr/glenda d-rwxrwxr-x arisawa sys 0 2015/12/25 09:25:52 grid/ns/usr/none --rw-rw-r-- arisawa sys 89 2015/12/24 13:29:45 grid/patt term%
where glenda is merely an example. and grid/patt is a pattern file for cpu command.
The content is:
- /mnt/factotum - /mnt/wsys/(.*/)?(text|screen|window)
And have a new cpu command, for example sgcpu (safe guard cpu) with the contents:
#!/bin/rc rfork ne cd /usr/none/lib/grid for(f in cons consctl draw random) bind /dev/$f ns/dev/$f for(f in bin net proc) bind /$f ns/$f bind -c '#e' ns/env bind -a /mnt/factotum ns/mnt/factotum bind /mnt/wsys ns/mnt/wsys bind -a /usr/none ns/usr/none bind -a /usr/glenda ns/usr/glenda ## example bind ns / cd /usr/none /bin/cpu -P lib/grid/patt $*
Excute
sgcpu -h grid.nyx.link -k 'dom=outside.plan9.bell-labs.com'
grid% kfind usage: kfind -t database pattern db is sys, rfc example: kfind2 -t rfc esmtp |p kfind2 -t sys 'exe*n' |p kfind2 -t sys 'exe*n&fork' |p kfind2 -t sys 'exe*n|fork' |p kfind2 -t sys -t rfc 'subm*&smtp' |p # Two types of wild card, "*" and "?", are allowed. # In using wild card, they must follow after two or more letters. you may use databases listed below: rfc sources sys grid%
grid% kdict kdict 'query here' |p # Eijiro kdict -d ryaku 'query here' |p # Ryakujiro kdict -d war 'query here' |p # Tolstoy: War and peace grid%
The dictionary database is based on Kirara. So same pattern matching rule is applied.
If you want to know your IP address on Internet, then try to access
grid.nyx.linkluna.nyx.linkluna.nyx.link is the authentication server for grid.nyx.link.2402:6b00:4040:b600::9!8006 2402:6b00:4040:b600:22c9:d0ff:fe8b:2b5!56645where the first line is IP address and port number of server, the second line is those of client.
You can access using your web browser with URL
http://grid.nyx.link:8006
http://luna.nyx.link:8007
On unix, you can do
telnet [-46] grid.nyx.link 8006 telnet [-46] luna.nyx.link 8007
telnet tcp!grid.nyx.link!8006 telnet tcp!luna.nyx.link!8007
connect is better for this purpose.connect [-6] tcp!grid.nyx.link!8006 connect [-6] tcp!luna.nyx.link!8007
connect from http://p9.nyx.link/netlib/cmd/connect/ .
The cpu command does not have IPv6 option.
Use literal IPv6 address for the moment, the fix is easy though.
If you find a problem, email me please.
none
Host owner and user none are special users of Plan9.
Other users are devided in two categories:
Host owner can do everything.
User none is for network services.
Grid users are restricted in small subset of namespace.
For any users, mount/unmount operation is disabled after “rfork m”.
The following operations are disabled for regular users
The following operations are disabled for grid users
Ramdisk is provided for grid users.
Any user (including grid user) can read/write files in client side.
Known problem with process creation is fixed as follows:
A user named alice on Terminal wants to connect to a CPU server named HOST with factotum key:
key proto=p9sk1 dom=DOMa user=IDu !password=PASSu
IDu is a user name, say alice, on a authentication domain DOMa and PASSu is a password of IDu on DOMa. The command may becpu -h HOST -k 'dom=DOMa'
We assume the CPU server is waiting connections with a factotum key:
key proto=p9sk1 dom=DOMa user=IDa !password=PASSa
IDa is is a user, say bob, on DOMa and PASSa is a password of IDa on DOMa.
Other possible players might be:
IDs: host owner of serverIDc: host owner of client (terminal)IDr: a user that IDc want to be in the serverIDc != IDr and also may be IDs != IDa and/or IDc != IDu.
Note that: if DOMa is not our domain then we have security problem. For example, IDr may be a user ID for Carol. Therefor cross-domain authentication needs extra special attribute such as grid and the created processes need special owner ID such as IDu@DOMa.
The user IDs must be hostid of the CPU server machine. (look /lib/ndb/auth for hostid.)
In regular situation HOST is a server in domain DOMa. Then the authentication mechanism is well explained in the paper by Russ et al[5].
However we want to consider cross-domain authentication.
For example, HOST is grid.nyx.link and DOMa is outside.bell-labs.com.
Then my question is: how grid.nyx.link can convince that alice is a user in outside.bell-labs.com?
The paper is misleading. Probably the paper is old for understanding cross-domain authentication. Reading the factotum code, you will find that the real processes are essentially:
C → S: CHcS → C: CHs,DOMa,IDaC → A: CHs,DOMa,IDa,IDuA → C: Ku{CHs,IDu,Kn}, Ka{CHs,IDu,Kn}C → S: Ka{CHs,IDu,Kn}, Kn{CHs,rand}S → C: Kn{CHc,rand}C is a clientS is a serverA is an authentication server for DOMaCHc is a challenge generated by the clientCHs is a challenge generated by the serverKn is a random value called nonce.Ku{CHs,IDu,Kn} is a message encrypted with Ku (IDu's key)Ka{CHs,IDu,Kn} is a message encrypted with Ka (IDa's key)Kn{CHs,rand} and Kn{CHc,rand} are messages encrypted with Knrand is a random value only for replay protection (referred as counter in the paper )A knows both Ku and KaKuKa
The set of data (CHs,IDu,Kn) is called ticket.
The value Kn is used for starting SSL encrypted channel.
[5] Russ Cox, Eric Crosse, Rob Pike, Dave Presotto and Sean Quilan: Security in Plan 9
http://plan9.bell-labs.com/sys/doc/auth.pdf
http://p9.nyx.link/labs/sys/doc/auth.pdf (mirror)