We spent the last two days of 2019 participating in a hackathon hosted by Prodigy Education, where many teams came together to try to make their ideas a reality.
Some of us just want to have fun, some of us want to learn something new, and some of us may want to prove concepts or ideas.
I’ve been passively retrieving Rust information or using Rust code for the past few weeks, so I thought hackathon would be a great time to learn about Rust.
The time urgency of Hackathon allows me to learn more quickly and solve real-world problems at the same time.
Why Rust
I spent 8 of the first 10 years of my career working with C and C++.
On the plus side, I like languages like C++ that provide static typing because it can catch errors early at compile time.
My personal views on C++ are:
- It’s easy for engineers to shoot themselves in the foot
- As a programming language, it is already bloated and complex
- Lack of a good, standard, widely applicable package management system
Since I switched to Web apps, IT has been Python and JavaScript development, using frameworks like Django, Flask, and Express.
My experience so far in Python and JavaScript development is that they can provide good program iteration and delivery speed, but can sometimes take up a lot of CPU and memory, even when the services are relatively idle.
I often find myself writing C++ programs that lack security, speed, and simplicity.
I was looking for a lean, bare-metal programming language like Rust to develop Web applications.
No runtime, no garbage collection. Load the binaries directly and hand them over to the kernel for execution.
The target
My goal is to create an application with a back end written by Rust and a front end done by JavaScript+React similar to S3 as a graph bed, where users can do the following:
- Browse all the pictures in the bed (paging optional)
- To upload pictures
- You can tag images when uploading them
- Query or filter by name
All interesting Hackathon projects have a name, so I decided to name this one:
RustIC -> Rust + Image Contents
I think hackathon will be a success for me if I do the following things:
- There is a basic understanding of Rust, including its type system and memory model
- Explore S3’s pre-signed links for files and arbitrary tags
- Write a functional application that can be verified
Since my main goal is to develop features and learn at the same time. I wrote a lot of the code as I learned it, so code organization and efficiency may not be optimal because they are secondary goals.
The principle of Rust
Before I begin, I was intrigued to find out what principles the designer of the language to learn had in mind when creating the language. I found a simplified version and a detailed version.
Contrary to what I’ve read on many blogs, Rust is prone to memory leaks (cyclic references) and unsafe operations (in the unsafe code block), as detailed in the FAQ above.
“We [the language Creators] do not intend [for Rust] to be 100 percent static, 100 percent safe, 100 percent creators.”
Start at the back end
Google for “Rust Web Framework” and Rocket tops the list. I went to the site and found examples of documentation at a glance.
One thing to note is that Rocket requires the Nightly version of Rust, but on Hackathon these are minor issues.
GitHub’s code base is rich with examples. Perfect!
I created a new project using Cargo, added Rocket dependencies to the TOML file, and then followed the Rocket starter guide to write the first piece of code:
#[get("/")]
fn index() - > &'static str {
"Hello, world!"
}
fn main() {
rocket::ignite().mount("/", routes! [index]).launch(); }Copy the code
For those of you familiar with Django, Flask, Express, etc., this code is very easy to read. As a Rocket user, you can use macros as decorators to map routes to the corresponding handlers.
At compile time, macros are extended. This is completely transparent to developers. If you want to see the expanded code, you can use cargo expand.
Here are some interesting or challenging highlights of my Rust application building:
Specifying route Response
I want to return a list of all the files in S3 in JSON data format.
You can see that the code of the route association handler determines the response type.
Setting up the response structure is very easy. If you want to return data in JSON format, and each field has its own structure and type, that corresponds to Rust’s struct.
So you should first define a struct(S) to receive the response and annotate it:
#[derive(Serialize)]
Copy the code
Struct (s) is marked with #[derive(Serialize)], so it can be converted to JSON via ‘Rocket_contrib ::json:: json.
#[derive(Serialize)]
struct BucketContents {
data: Vec<S3Object>,
}
#[derive(Serialize)]
struct S3Object {
file_name: String,
presigned_url: String,
tags: String,
e_tag: String.// AWS generated MD5 checksum hash for object
is_filtered: bool,}#[get("/contents?
"
)]
fn get_bucket_contents(
filter: Option<&RawStr>
) -> Result<Json<BucketContents>, Custom<String> > {// Returns either Ok(Json(BucketContents)) or,
// a Custom error with a reason
}
Copy the code
Handling segmented uploads
When I realized that my front end would most likely use POST to upload form data in the multipart/form-data format, I delved into Rocket to build applications.
Unfortunately, Rocket0.4 does not support multipart, and it looks like it will in 0.5.
This means I need to use Multipart Crate and integrate it into Rocket. As a result, the code will work fine, but it will be much cleaner if Rocket supports Multipart.
#[post("/upload", data = "<data>")]
// signature requires the request to have a `Content-Type`. The preferred way to handle the incoming
/ / data he have had to use the FromForm trait as described here: https://rocket.rs/v0.4/guide/requests/#forms
// Unfortunately, file uploads are not supported through that mechanism since a file upload is performed as a
// multipart upload, and Rocket does not currently (As of v0.4) support this.
// https://github.com/SergioBenitez/Rocket/issues/106
fn upload_file(cont_type: &ContentType, data: Data) -> Result<Custom<String>, Custom<String> > {// this and the next check can be implemented as a request guard but it seems like just
// more boilerplate than necessary
if! cont_type.is_form_data() {return Err(Custom(
Status::BadRequest,
"Content-Type not multipart/form-data".into()
));
}
let (_, boundary) = cont_type.params()
.find(|&(k, _)| k == "boundary")
.ok_or_else(
|| Custom(
Status::BadRequest,
"`Content-Type: multipart/form-data` boundary param not provided".into()
)
)?;
// The hot mess that ensues is some weird combination of the two links that follow
// and a LOT of hackery to move data between closures.
// https://github.com/SergioBenitez/Rocket/issues/106
// https://github.com/abonander/multipart/blob/master/examples/rocket.rs
let mut d = Vec::new();
data.stream_to(&mut d).expect("Unable to read");
let mut mp = Multipart::with_body(Cursor::new(d), boundary);
let mut file_name = String::new();
let mut categories_string = String::new();
let mut raw_file_data = Vec::new();
mp.foreach_entry(|mut entry| {
if *entry.headers.name == *"fileName" {
let file_name_vec = entry.data.fill_buf().unwrap().to_owned();
file_name = from_utf8(&file_name_vec).unwrap().to_string()
} else if *entry.headers.name == *"tags" {
let tags_vec = entry.data.fill_buf().unwrap().to_owned();
categories_string = from_utf8(&tags_vec).unwrap().to_string();
} else if *entry.headers.name == *"file" {
raw_file_data = entry.data.fill_buf().unwrap().to_owned()
}
}).expect("Unable to iterate");
let s3_file_manager = s3_interface::S3FileManager::new(None.None.None.None);
s3_file_manager.put_file_in_bucket(file_name.clone(), raw_file_data);
let tag_name_val_pairs = vec![("tags".to_string(), categories_string)];
s3_file_manager.put_tags_on_file(file_name, tag_name_val_pairs);
return Ok(
Custom(Status::Ok."Image Uploaded".to_string())
);
}
Copy the code
Configuration CORS
Once the route is written, I’ll start testing it with curl or Postman, and now it’s time to start integrating the front end. I need to set up the response header properly to avoid cross-domain issues.
Rocket still doesn’t support this feature.
Then I found some solutions in the GitHub codebase:
// CORS Solution below comes from: https://github.com/SergioBenitez/Rocket/issues/25
extern crate rocket;
use std::io::Cursor;
use rocket::fairing::{Fairing, Info, Kind};
use rocket::{Request, Response};
use rocket::http::{Header, ContentType, Method};
struct CORS(a);impl Fairing for CORS {
fn info(&self) -> Info {
Info {
name: "Add CORS headers to requests",
kind: Kind::Response
}
}
fn on_response(&self, request: &Request, response: &mut Response) {
if request.method() == Method::Options ||
response.content_type() == Some(ContentType::JSON) ||
response.content_type() == Some(ContentType::Plain) {
response.set_header(Header::new("Access-Control-Allow-Origin"."http://localhost:3000"));
response.set_header(Header::new("Access-Control-Allow-Methods"."POST, GET, OPTIONS"));
response.set_header(Header::new("Access-Control-Allow-Headers"."Content-Type"));
response.set_header(Header::new("Access-Control-Allow-Credentials"."true"));
}
if request.method() == Method::Options {
response.set_header(ContentType::Plain);
response.set_sized_body(Cursor::new("")); }}}fn main() {
rocket::ignite().attach(
CORS()
).mount(
"/", routes! [get_bucket_contents, upload_file] ).launch(); }Copy the code
After a while, I found Rocket_Cors, which helped me drastically reduce the amount of code.
fn main() - >Result<(), Error> {
let allowed_origins = AllowedOrigins::some_exact(&["http://localhost:3000"]);
let cors = rocket_cors::CorsOptions {
allowed_origins,
allowed_methods: vec![Method::Get, Method::Post].into_iter().map(From::from).collect(),
allowed_headers: AllowedHeaders::some(&["Content-Type"."Authorization"."Accept"]),
allow_credentials: true.Default: :default() } .to_cors()? ; rocket::ignite().attach(cors) .mount("/", routes! [get_bucket_contents, upload_file]) .launch();Ok(())}Copy the code
To run
A simple cargo run command is all we need to get the program running
The activity monitor on my machine tells me that the program is running and consuming only 2.7MB of memory.
And this is just a debug version that hasn’t been optimized. A project packaged with the -release tag only needs 1.6MB of ram at runtime.
On a Rust backend server, we request /contents for this route and get the following response:
{
"data": [{"file_name": "Duck.gif"."presigned_url": "https://s3.amazonaws.com/rustic-images/Duck.gif?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIARDWJNDW3U8329UDNJ %2F20200107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200107T050353Z&X-Amz-Expires=1800&X-Amz-Signature=1369c003b2f54 510882bf9982ab56d024d6c9d2655a4d86f8907313c7499b56d&X-Amz-SignedHeaders=host"."tags": "animal"."e_tag": "\"93c570cadd6b8b2f85b47c2f14fd82a1\""."is_filtered": false
},
{
"file_name": "GIZMO.png"."presigned_url": "https://s3.amazonaws.com/rustic-images/GIZMO.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIARDWJNDW3U8329UDN J%2F20200107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200107T050353Z&X-Amz-Expires=1800&X-Amz-Signature=040e76c2df5a 9a54ed4fbc8490378cf732b32bae78f628448536fc610018c0c3&X-Amz-SignedHeaders=host"."tags": "robots"."e_tag": "\"2cde221a0c7a72c0a7a60cffce29a0bc\""."is_filtered": false
},
{
"file_name": "GreenSmile.gif"."presigned_url": "https://s3.amazonaws.com/rustic-images/GreenSmile.gif?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIARDWJNDW3U83 29UDNJ%2F20200107%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200107T050354Z&X-Amz-Expires=1800&X-Amz-Signature=d115b10 7de530ce15b3590abdbab355c2a9481a81131f88bf4ad2a59ca11bbac&X-Amz-SignedHeaders=host"."tags": "smile-face"."e_tag": "\"86854a599540f50bdc5e837d30ca34f9\""."is_filtered": false}}]Copy the code
The front-end work is relatively simple, we use:
- React
- React Bootstrap
- react-grid-gallery
- react-tags-input
Users can browse pictures on our page, or search or filter them by file name or label.
Users can also drag and drop files and tag them before submitting them for upload.
What I like about building applications with Rust
- Cargo’s level of dependency and application management is amazing
- The compiler has been very helpful in handling compilation errors, and a blogger described how he wrote code following the compiler’s instructions. My experience is similar.
- I was pleasantly surprised to find crate for every feature I needed
- Rust Playground, online, allows me to run small snippets of code.
- The Rust language server, which is well integrated with Visual Studio Code, provides real-time error checking, formatting, symbol lookup, and more. This allowed me to make good progress in a few hours without compiling.
Inconvenience, surprise and trouble
While Rust’s documentation is great, I had to rely on some of Crates’ documentation and examples. Some Crates have great integration tests that provide hints on how to use them. Of course, Stack Overflow and Reddit have also helped me a lot.
Also note:
- Understanding ownership, lifecycle, and ownership borrowing can make learning difficult, especially when trying to deliver functionality during a two-day hackathon. I compared them to C++ and figured it out, but sometimes I still got confused.
- Of all things,
Strings
Stopped me for a few minutes, especiallyString
and&str
The distinction is even more confusing — until I spent some time understanding ownership, life cycles, and ownership borrowing.
Some other observations
- There is no true null type in Rust; in general, null values are needed
Option
The type ofNone
To represent the - Pattern matching is great; it’s one of my favorite features in Scala, and it’s the same in Rust. This code looks expressive and allows the compiler to flag unhandled cases.
match bucket_contents {
Err(why) => match why {
S3ObjectError::FileWithNoName => Err(Custom(
Status::InternalServerError,
"Encountered bucket objects with no name".into()
)),
S3ObjectError::MultipleTagsWithSameName => Err(Custom(
Status::InternalServerError,
"Encountered a file with a more than one tag named 'tags'".into()
))
},
Ok(s3_objects) => {
let visible_s3_objects: Vec<S3Object> = s3_objects.into_iter() .filter(|obj| ! obj.is_hidden()) .collect();Ok(Json(BucketContents::new(visible_s3_objects)))
}
}
Copy the code
- Speaking of secure and insecure modes, you can still do lower-level programming, such as interfacing with C code in insecure mode. Despite the many correctness checks in Rust, you can still do things in insecure modules, such as dereferencing. Readers of code can also learn a lot from insecure modules.
- through
Box
Allocate memory space in the heap instead ofnew
anddelete
. It felt strange at first, but it was easy to understand. Others are defined in the librarySmart PointersIf you need to use the number of references or weak references, you can use them directly. - Exceptions in Rust are also interesting because there are no exceptions. You can choose to use it
Result<T, E>
An error that can be recoveredpanic!
Macros represent unrecoverable errors.
// This code:
// 1. Takes a vector of objects representing S3 contents
// 2. Uses filter to remove entries we don't care about
// 3. Uses map to transform each object into another type, but terminates iteration
// . if the lambda passed to map returns an Err.
// 4. If all iterations produced an Ok(S3Object) result, these are collected into a Vec<S3Object>
let bucket_contents: Result<Vec<S3Object>, S3ObjectError> = bucket_list
.into_iter()
.filter(|bucket_obj| bucket_obj.size.unwrap_or(0) != 0) // Eliminate folders
.map(|bucket_obj| {
if let None = bucket_obj.key {
return Err(S3ObjectError::FileWithNoName);
}
let file_name = bucket_obj.key.unwrap();
let e_tag = bucket_obj.e_tag.unwrap_or(String::new());
let tag_req_output = s3_file_manager.get_tags_on_file(file_name.clone());
let tags_with_categories: Vec<Tag> = tag_req_output.into_iter()
.filter(|tag| tag.key == "tags")
.collect();
if tags_with_categories.len() > 1 {
return Err(S3ObjectError::MultipleTagsWithSameName);
}
let tag_value = if tags_with_categories.len() == 0 {
"".to_string()
} else {
tags_with_categories[0].value.clone()
};
let presigned_url = s3_file_manager.get_presigned_url_for_file(
file_name.clone()
);
Ok(S3Object::new(
file_name,
e_tag,
tag_value,
presigned_url,
false,
))
})
.collect();
Copy the code
Here’s how the manual describes it:
In most cases, Rust requires you to know as much about errors as you can and to handle them before compiling. This requirement makes your application more robust and ensures that you can find and handle bugs before you release them.
Key points and Lessons
- John Carmack once described the experience of writing Rust as “very rewarding.” I agree with this feeling, the hackathon felt like opening the door to a new world and discovering a lot of new things, not just at the code level.
- In hindsight, I should have chosen the network framework more carefully. With a little more thought, I might have taken a different path. I’ll probably go for iron, Actix-Web, or tiny-HTTP next time.
- I’ve only scratched the surface of Rust. It’s impossible to become a Rustacean in 16 hours, even if I’m curious about the language and do some digging. I’m excited about the future of Rust. I think it brings a lot of discipline to building applications. It’s a very expressive language that offers speed and memory performance comparable to C++.
resources
RustIC backend code
RustIC front-end code
Rusoto: An AWS SDK for Rust
The original link
Medium.com/better-prog…