Rust + LLVM ORC JIT cannot find symbol address - rust

I have been trying to use the ORC JIT compiler from the LLVM C bindings in Rust, but I keep running into the problem that LLVMOrcGetSymbolAddress is not able to find the symbol of my function run in the module I provide it. The code below combines the most important parts of my code that unfortunately doesn't work. All goes well until the last part of foo, where an error is returned because the function cannot be found. The function is definitely in the module, as LLVMGetNamedFunction is able to find it, but somehow the ORC engine cannot see it. Can anyone see what I am doing wrong? I am using LLVM 6.0 and the llvm-sys Rust bindings. Everything works fine if I use MCJIT but I need ORC for lazy compilation.
fn foo(module: LLVMModuleRef) -> Result<I64Func, LlvmError> {
let def_triple = LLVMGetDefaultTargetTriple();
let mut target_ref = ptr::null_mut();
let mut error_str = ptr::null_mut();
// Get target from default triple
if LLVMGetTargetFromTriple(def_triple, &mut target_ref, &mut error_str) != 0 {
let msg = format!("Creating target from triple failed: {}", CStr::from_ptr(error_str).to_str().unwrap());
LLVMDisposeMessage(def_triple);
LLVMDisposeMessage(error_str);
return Err(LlvmError(msg));
}
// Check if JIT is available
if LLVMTargetHasJIT(target_ref) == 0 {
let msg = format!("Cannot do JIT on this platform");
LLVMDisposeMessage(def_triple);
return Err(LlvmError(msg));
}
// Create target machine
let tm_ref = LLVMCreateTargetMachine(target_ref,
def_triple,
CString::new("").unwrap().as_ptr(),
CString::new("").unwrap().as_ptr(),
llvm_opt_level(optimization_level)?,
LLVMRelocMode::LLVMRelocDefault,
LLVMCodeModel::LLVMCodeModelJITDefault);
LLVMDisposeMessage(def_triple);
let engine = LLVMOrcCreateInstance(tm_ref);
// Add eagerly compiled IR
let mut handle = LLVMOrcModuleHandle::default();
let shared_module = LLVMOrcMakeSharedModule(module);
let ctx = engine as *mut libc::c_void;
map_orc_err(engine, LLVMOrcAddEagerlyCompiledIR(engine,
&mut handle,
shared_module,
symbol_resolver_callback,
ctx))?;
// Find function named 'run'
let c_name = CString::new("run").unwrap().as_ptr();
let mut func_addr = LLVMOrcTargetAddress::default();
map_orc_err(engine, LLVMOrcGetSymbolAddress(engine, &mut func_addr, c_name))?;
if func_addr == 0 {
// This errors always gets thrown
return Err(LlvmError(format!("No function named {} in module", name)));
}
let function: I64Func = mem::transmute(func_addr);
Ok(function)
}
extern "C" fn symbol_resolver_callback(symbol: *const libc::c_char, ctx: *mut libc::c_void) -> LLVMOrcTargetAddress {
let mut address = LLVMOrcTargetAddress::default();
let engine: LLVMOrcJITStackRef = ctx as LLVMOrcJITStackRef;
unsafe { LLVMOrcGetSymbolAddress(engine, &mut address, symbol) };
address
}
unsafe fn map_orc_err(engine: LLVMOrcJITStackRef, error_code: LLVMOrcErrorCode) -> Result<(), LlvmError> {
match error_code {
LLVMOrcErrorCode::LLVMOrcErrSuccess => Ok(()),
LLVMOrcErrorCode::LLVMOrcErrGeneric => {
let c_str: &CStr = CStr::from_ptr(LLVMOrcGetErrorMsg(engine));
let str_slice: &str = c_str.to_str().unwrap();
let str_buf: String = str_slice.to_owned();
Err(LlvmError(str_buf))
}
}
}
EDIT: I tried downgrading to LLVM 4.0 just to see what effect that might have. I still cannot resolve the function, but now I'm getting an assertion error:
Assertion failed: (!Name.empty() && "getNameWithPrefix requires non-empty name"), function getNameWithPrefixImpl, file /tmp/llvm-4.0-20180412-49671-1pw0nxu/llvm-4.0.1.src/lib/IR/Mangler.cpp, line 37.
EDIT 2: Below is some basic IR for which the engine fails to find the function address:
define i64 #bar(i64 %arg) {
%1 = add i64 %arg, 1
ret i64 %1
}
define i64 #run(i64 %arg) {
%1 = add i64 %arg, 1
%2 = call i64 #bar(i64 %1)
ret i64 %2
}

Related

OpenCV different results on Windows and Linux

I'm writing a cross-platform wrapper for OpenCV on rust. And I wrote several tests to check that my wrapper is working correctly. Some tests are passing, some tests are failing with slightly different values, but one test has completely different results.
Right number is what I get on windows, left - on linux
---- compare_hist_chi_square stdout ----
thread 'compare_hist_chi_square' panicked at '2697.981816285168 == 1360.7', tests/test_imgproc.rs:81:4
---- compare_hist_bhattacharyya stdout ----
thread 'compare_hist_bhattacharyya' panicked at '0.6798259690477988 == 0.6679', tests/test_imgproc.rs:81:4
---- compare_hist_chi_square_alternative stdout ----
thread 'compare_hist_chi_square_alternative' panicked at '41.65613074156445 == 41.027', tests/test_imgproc.rs:81:4
---- compare_hist_correlation stdout ----
thread 'compare_hist_correlation' panicked at '0.20456957916644988 == 0.211', tests/test_imgproc.rs:81:4
---- compare_hist_intersection stdout ----
thread 'compare_hist_intersection' panicked at '5.440850785933435 == 5.682', tests/test_imgproc.rs:81:4
---- compare_hist_kullback_leibler_divergence stdout ----
thread 'compare_hist_kullback_leibler_divergence' panicked at '55.71912075710992 == 54.06287', tests/test_imgproc.rs:81:4
I tried to reproduce code from this article. I took same images and wanted to get same results. However, it didn't happen.
Here is some Rust code that tests (just for reference):
extern crate cv;
extern crate float_cmp;
mod utils;
use cv::*;
use cv::imgproc::*;
use float_cmp::ApproxEqRatio;
use utils::*;
#[test]
#[should_panic]
fn compare_hist_different_dimensions_panic() {
let first_image = load_unchanged("assets/Histogram_Comparison_Source_0.jpg");
let second_image = load_unchanged("assets/Histogram_Comparison_Source_1.jpg");
let _ = first_image.compare_hist(&second_image, HistogramComparisionMethod::Corellation).unwrap();
}
#[test]
fn compare_hist_correlation() {
compare_hist(HistogramComparisionMethod::Corellation, 0.211);
}
#[test]
fn compare_hist_chi_square() {
compare_hist(HistogramComparisionMethod::ChiSquare, 1360.7);
}
#[test]
fn compare_hist_intersection() {
compare_hist(HistogramComparisionMethod::Intersection, 5.682);
}
#[test]
fn compare_hist_bhattacharyya() {
compare_hist(HistogramComparisionMethod::Bhattacharyya, 0.6679);
}
#[test]
fn compare_hist_chi_square_alternative() {
compare_hist(HistogramComparisionMethod::ChiSquareAlternative, 41.027);
}
#[test]
fn compare_hist_kullback_leibler_divergence() {
compare_hist(
HistogramComparisionMethod::KullbackLeiblerDivergence,
54.06287,
);
}
fn compare_hist(method: HistogramComparisionMethod, expected_result: f64) {
let first_image = get_image_histogram("assets/Histogram_Comparison_Source_0.jpg");
let second_image = get_image_histogram("assets/Histogram_Comparison_Source_1.jpg");
let result = first_image.compare_hist(&second_image, method).unwrap();
assert_eq(result, expected_result);
}
fn get_image_histogram(path: &'static str) -> Mat {
let image = load_unchanged(path);
let image = image.cvt_color(ColorConversionCodes::BGR2HSV);
let hsize = [50, 60];
let h_ranges = [0_f32, 180_f32];
let s_ranges = [0_f32, 256_f32];
let ranges = [
h_ranges.as_ptr() as *const f32,
s_ranges.as_ptr() as *const f32,
];
let channels = [0, 1];
let image = image.calc_hist(
channels.as_ptr(),
Mat::new(),
2,
hsize.as_ptr(),
ranges.as_ptr(),
);
let image = image.normalize(0_f64, 1_f64, NormTypes::NormMinMax);
image
}
fn assert_eq(a: f64, b: f64) {
assert!(a.approx_eq_ratio(&b, 0.001), format!("{} == {}", a, b));
}
pub fn load_unchanged<P: AsRef<Path>>(img: P) -> Mat {
let buf = load_image_as_buf(img);
Mat::imdecode(&buf, ImreadModes::ImreadUnchanged)
}
fn load_image_as_buf<P: AsRef<Path>>(img: P) -> Vec<u8> {
let mut d = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
d.push(img);
let mut buf = Vec::new();
File::open(d).unwrap().read_to_end(&mut buf).unwrap();
buf
}
Probably it's because of jpg and different jpg readers on different platforms, but it can't explain 2697.98 vs 1360.7, it's almost 2x difference!
What could be wrong here? Here is my entire pull request where I'm trying to add this functionality (careful, some Rust code is included)
Thanksto #VTT, this issue happens because of different jpg interpretation on different platforms. Switch to png solves the problem

Is there some way that rust-cargo put DLL file into exe program?

I write a tool, It can get device id through the dll, and copy the id to clipboard.
extern crate clipboard;
use clipboard::ClipboardProvider;
use clipboard::ClipboardContext;
extern crate libloading;
use libloading::{Library, Symbol};
use std::ffi::CStr;
use std::str;
type GetHylinkDeviceId = unsafe fn() -> *const i8;
fn copy_to_clipboard(x3id: String) {
let mut ctx: ClipboardContext = ClipboardProvider::new().unwrap();
println!("{:?}", ctx.get_contents());
ctx.set_contents(x3id);
}
fn get_x3id() -> String {
let library_path = "PcInfo.dll";
println!("Loading add() from {:?}", library_path);
let lib = Library::new(library_path).unwrap();
let x3id = unsafe {
let func: Symbol<GetHylinkDeviceId> = lib.get(b"GetHylinkDeviceId").unwrap();
let c_buf: *const i8 = func();
let c_str: &CStr = unsafe { CStr::from_ptr(c_buf) };
let str_slice: &str = c_str.to_str().unwrap();
let str_buf: String = str_slice.to_owned();
str_buf
};
x3id
}
fn main() {
let x3id: String = get_x3id();
println!("{:?}", x3id);
copy_to_clipboard(x3id)
}
It works well, I use cargo build it, auto generate an executable. I need to put dll file and exe in the same dir when run it.
I want to know if there is anyway to pack dll file into the exe by cargo?

Detecting client hangup in MIO

When using MIO (0.3.5) how do I detect the termination of a connection?
I tried the following:
extern crate mio;
use mio::{EventLoop,Token,ReadHint};
use std::io::Read;
fn main(){
let listener = mio::tcp::TcpListener::bind("localhost:1234").unwrap();
let (stream,_) : (mio::tcp::TcpStream, _) = listener.accept().unwrap();
let mut event_loop = EventLoop::new().unwrap();
event_loop.register(&stream,Token(0)).unwrap();
println!("run...");
event_loop.run(&mut H{stream:stream}).unwrap();
}
struct H{stream : mio::tcp::TcpStream}
impl mio::Handler for H{
type Timeout = ();
type Message = ();
fn readable(&mut self, _ : &mut EventLoop<Self>, _ : Token, hint: ReadHint){
let mut buf: [u8; 500] = [0; 500];
println!("{} {}",(hint==ReadHint::data()),self.stream.read(&mut buf).unwrap());
std::thread::sleep_ms(1000);
}
}
Run this. Connect to it using something like nc localhost 1234. Terminate connection by using Ctrl-C. My code will think there is new data available (hint==ReadHint::data()). A try to read it will result in zero bytes available.
Shouldn't the hint be something like ReadHint::hup()?
The default when calling register on mio v0.3.5 is to register only for the readable Interest, so that is the only hint you will get.
If you want to be warned for hup as well, you need to use the function register_opt and give your Interest and PollOpt as parameters, so your code becomes:
extern crate mio;
use mio::{EventLoop,Token,ReadHint,PollOpt,Interest};
use std::io::Read;
fn main() {
let listener = mio::tcp::TcpListener::bind("localhost:1234").unwrap();
let (stream,_) : (mio::tcp::TcpStream, _) = listener.accept().unwrap();
let mut event_loop = EventLoop::new().unwrap();
let interest = Interest::readable() | Interest::hup();
event_loop.register_opt(&stream,Token(0), interest,PollOpt::level()).unwrap();
println!("run...");
event_loop.run(&mut H{stream:stream}).unwrap();
}
struct H{stream : mio::tcp::TcpStream}
impl mio::Handler for H {
type Timeout = ();
type Message = ();
fn readable(&mut self, event_loop : &mut EventLoop<Self>, _ : Token, hint: ReadHint){
let mut buf: [u8; 500] = [0; 500];
if hint.is_hup() {
println!("Recieved hup, exiting");
event_loop.shutdown();
return;
}
println!("{} {}",hint.is_data(),self.stream.read(&mut buf).unwrap());
std::thread::sleep_ms(1000);
}
}
I think the default for the convenience function register has changed in v0.4.0 to be Interest::all() so this shouldn't be a problem in the future.

Creating a vector of zeros for a specific size

I'd like to initialize a vector of zeros with a specific size that is determined at runtime.
In C, it would be like:
int main(void)
{
uint size = get_uchar();
int A[size][size];
memset(A, 0, size*size*sizeof(int));
}
Here's the helper function that I tried writing in Rust, but I think the slicing syntax 0..size is offending the compiler. Besides, it looks more verbose than the C version. Is there a more idiomatic way to do this?
fn zeros(size: u32) -> Vec<i32> {
let mut zero_vec: Vec<i32> = Vec::with_capacity(size);
for i in 0..size {
zero_vec.push(0);
}
return zero_vec;
}
I swear that the old docs used to explain a from_elem() method here and none of the permutations of the [0 ; size] notation seem to work
I'd like to stick this into a substring search algorithm ultimately:
pub fn kmp(text: &str, pattern: &str) -> i64 {
let mut shifts = zeros(pattern.len()+1);
}
To initialize a vector of zeros (or any other constant value) of a given length, you can use the vec! macro:
let len = 10;
let zero_vec = vec![0; len];
That said, your function worked for me after just a couple syntax fixes:
fn zeros(size: u32) -> Vec<i32> {
let mut zero_vec: Vec<i32> = Vec::with_capacity(size as usize);
for i in 0..size {
zero_vec.push(0);
}
return zero_vec;
}
uint no longer exists in Rust 1.0, size needed to be cast as usize, and the types for the vectors needed to match (changed let mut zero_vec: Vec<i64> to let mut zero_vec: Vec<i32>.
Here is another way, much shorter. It works with Rust 1.0:
fn zeros(size: u32) -> Vec<i32> {
vec![0; size as usize]
}
fn main() {
let vec = zeros(10);
for i in vec.iter() {
println!("{}", i)
}
}
You can also use the iter::repeat function, which I suppose is "more idiomatic" (and just looks nicer to me):
use std::iter;
fn zeros(size: usize) -> Vec<i32> {
iter::repeat(0).take(size).collect()
}
You may use resize
let mut v = Vec::new();
let l = 42;
v.resize(l, 0u32);

What's wrong with my std::ptr::copy_nonoverlapping call?

I've got a problem with my function save_vec that saves a vector down to disk using mmap. It creates the file with the right size but when I check its contents (using hexdump) it's just all zeros. The mmap is using MAP_SHARED and not private so assuming there's something wrong with my copy_nonoverlapping call.
fn save_vec<T: Encodable + Debug + Copy>(path: &str, v: &Vec<T>) -> Option<String> {
println!("save_vec");
let mut encoded: Vec<u8> = bincode::encode(v, SizeLimit::Infinite).unwrap();
let len = encoded.len();
println!("encoded = {:?}", encoded);
println!("len = {:?}", len);
let file = match OpenOptions::new()
.create(true)
.read(true)
.write(true)
.open(path) {
Ok(f) => f,
Err(err) => return Some("cannot create file".to_string())
};
file.set_len(len as u64);
let fd = file.as_raw_fd();
unsafe {
let mmap_len = len as libc::size_t;
let prot = libc::PROT_READ | libc::PROT_WRITE;
let flags = libc::MAP_SHARED;
let ptr = libc::mmap(0 as *mut libc::types::common::c95::c_void, mmap_len, prot, flags, fd, 0);
if ptr == libc::MAP_FAILED {
return Some("mmap fail".to_string())
}
let byte_ptr: *const u8 = mem::transmute(ptr);
let encoded_ptr: *mut u8 = encoded.as_mut_ptr();
ptr::copy_nonoverlapping(byte_ptr, encoded_ptr, len);
libc::munmap(ptr, len as u64);
println!("byte_ptr={:?}", byte_ptr);
}
None
}
You are probably being bitten by the last-minute change to the argument order of ptr::copy and friends. Specifically, this part of your code:
let byte_ptr: *const u8 = mem::transmute(ptr); // Why is this const?
let encoded_ptr: *mut u8 = encoded.as_mut_ptr(); // Why is this mutable?
ptr::copy_nonoverlapping(byte_ptr, encoded_ptr, len);
The compiler actually gave you some indication that something was wrong, in that you "needed" a mutable pointer to the source of data. That doesn't make any sense - we are just reading from that slice. And the destination was constant, which means we can't change it.
Here's the full code I ran that worked:
extern crate libc;
use std::{mem,ptr};
use std::fs::OpenOptions;
use std::os::unix::io::AsRawFd;
fn save_vec(path: &str, v: &[u8]) -> Result<(), String> {
let len = v.len();
let f =
OpenOptions::new()
.create(true)
.read(true)
.write(true)
.open(path);
let file = try!(f.map_err(|_| "cannot create file".to_string()));
file.set_len(len as u64).unwrap();
let fd = file.as_raw_fd();
unsafe {
let mmap_len = len as libc::size_t;
let prot = libc::PROT_READ | libc::PROT_WRITE;
let flags = libc::MAP_SHARED;
let ptr = libc::mmap(0 as *mut libc::types::common::c95::c_void, mmap_len, prot, flags, fd, 0);
if ptr == libc::MAP_FAILED {
return Err("mmap fail".to_string())
}
let byte_ptr: *mut u8 = mem::transmute(ptr);
let encoded_ptr: *const u8 = v.as_ptr();
ptr::copy_nonoverlapping(encoded_ptr, byte_ptr, len);
libc::munmap(ptr, len as u64);
}
Ok(())
}
fn main() {
save_vec("dump", b"hello").unwrap();
}

Resources