3

I'm a bit lost at making dlsym() work in Rust:

  • libc::dlsym() returns a *mut c_void handle to the desired function.
  • To use that handle, I need to cast it to extern fn, but that doesn't seem possible, even though fn (extern or not) appears to be a *mut internally.
  • If I store the handle as *mut extern fn, as I did in the sample program below, I'm effectively storing a *mut as a *mut *mut (see gdb output at the end), which, not surprisingly, will produce segfaults.

Sample program, which will segfault when handle is invoked at line 15:

use std::ffi::CString;

fn main() {
    let func_name = CString::new("fork").unwrap();

    let handle: *mut extern "C" fn() -> libc::pid_t =
        unsafe { libc::dlsym(libc::RTLD_NEXT, func_name.as_ptr()).cast() };

    if handle.is_null() {
        let error_msg = CString::new("dlsym(fork) failed: %s\n").unwrap();
        unsafe { libc::printf(error_msg.as_ptr(), libc::dlerror()) };
        panic!();
    }

    let pid = unsafe { (*handle)() };

    if pid == 0 {
        println!("parent pid = {}", unsafe { libc::getppid() });
    } else if pid == -1 {
        let error_msg = CString::new("fork() failed").unwrap();
        unsafe { libc::perror(error_msg.as_ptr()) };
    } else {
        println!("child pid = {}", pid);
    }
}

gdb output:

(gdb) p handle
$1 = (*mut *mut fn () -> i32) 0x7f40b093b900 <__libc_fork>
user869887
  • 198
  • 2
  • 6
  • @Frxstrem It does. I guess I didn't look hard enough--sorry! For future readers: one potential problem is that [std::mem::transmute](https://doc.rust-lang.org/stable/std/mem/fn.transmute.html) is "equivalent to `memcpy`" according to the official doc. I haven't verified that, but if it's true and you are shimming `memcpy()`, be ready to supply a backup implementation to avoid self-recursion. – user869887 Dec 01 '20 at 22:30

0 Answers0