Skip to content

README.md

Latest commit

 

History

History
66 lines (59 loc) · 2.17 KB

README.md

File metadata and controls

66 lines (59 loc) · 2.17 KB
Raw

rpc

cs 739 intro to communication

Install

  1. Protobuf
    1. git clone https://github.com/google/protobuf.git
    • configure --prefix=$HOME/local
    • make
    • make check
    • make install
  • Go Configuration
    • go get -a github.com/golang/protobuf/protoc-gen-go
    • go get google.golang.org/grpc
  • git clone https://github.com/grpc/grpc.git
    1. git submodule update --init
    • make
    • make install prefix=$HOME/local
  • ensure protoc-gen-go and grpc_cpp_plugin are present

Build

  • client
    1. gRPC Support cd ~/project2/proto; protoc --go_out=plugins=grpc:. *.proto
    • cd ~/project2/client; go build

[Crash-Consistent] (https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-pillai.pdf)

  • break down categories:
    • atomicity
    • ordering of operations: does file system ensure file creations are persisted in the same order?

FS operations major categories

  • file overwrite
  • file append
  • directory operations
    • rename
    • link
    • unlink
    • mkdir

[Presentation Video] (https://www.usenix.org/conference/osdi14/technical-sessions/presentation/pillai)

  • WriteAheadLog

  • ext3(data-ordered) need fsync to ensure order, prevent FS re-order

      creat(/x/log1);
  write(/x/log1, "2, 3, checksum, foo"); // checksum is to prevent garbage, which means file size alone increases
  fsync(/x/log1);
  fsync(/x); // ensure log file's directory entry might never be created
  pwrite(/x/f1, 2, "bar");
  fsync(/x/f1);
  unlink(/x/log1);
  fsync(/x); // additional fsync() required for durability in all FS

Our approach

  • Ensure crash-consistent inside write()
    1. compute file checksum
    • write checksum: 32 bits, filecontent into cache/filepath_base64
      • if cache file exists, over-write it
    • fsync() cache file into file system
    • we may add a hook here to cause system exit before write into FUSE
      • goroutine sends a channel signal
    • actual write into FUSE
    • delete cache file
    • return
  • crash recovery upon startup
    1. see whether there are files inside cache folder
    • decode files and apply to FUSE
    • start normal service