Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CRASH] segfault in dialog.so #3494

Open
MarcoHierl opened this issue Oct 14, 2024 · 1 comment
Open

[CRASH] segfault in dialog.so #3494

MarcoHierl opened this issue Oct 14, 2024 · 1 comment
Assignees
@bogdan-iancu
Labels
bug

Comments

@MarcoHierl
Copy link

OpenSIPS version you are running

opensips@sbc-test-hh$ /usr/opensips/akt/sbin/opensips -V
version: opensips 3.4.8 (x86_64/linux)
flags: STATS: On, SHM_EXTRA_STATS, DISABLE_NAGLE, USE_MCAST, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, HP_MALLOC, DBG_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16, MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll, sigio_rt, select.
git revision: 11c99480a
main.c compiled on 14:17:57 Oct  9 2024 with gcc 12

Crash Core Dump
can be send if needed

Describe the traffic that generated the bug
happened during normal operation

Relevant System Logs

CRITICAL:core:sig_usr: segfault in process pid

OS/environment information

  • Operating System: Debian GNU/Linux 12 (bookworm)
  • OpenSIPS installation: git clone --recurse-submodules https://github.com/OpenSIPS/opensips.git -b 3.4
  • other relevant information: runs stable for 2 days after upgrade from 3.2.11, afterwards segfaults came one after the other, even after restart/reboot

Additional context

opensips@sbc-test-hh:/var/log/opensips/coredump$ gdb /usr/opensips/akt/sbin/opensips --core core-opensips-sig11-user1000-group1000-pid1109210-time1728634922
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/opensips/akt/sbin/opensips...

warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 1109210]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/opensips/akt/sbin/opensips -P /var/run/opensips/opensips.pid -f /usr/opens'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f3b8ef0568a in fetch_dlg_value (dlg=0x7f3b90508f78, name=name@entry=0x7f3bcf93c790, type=type@entry=0x7fffe6215a0c, out_val=out_val@entry=0x7fffe6215a10, val_has_buf=val_has_buf@entry=1)
    at dlg_vals.c:205
205                                     memcpy( val->s, dv->val.s.s, dv->val.s.len );
(gdb) bt full
#0  0x00007f3b8ef0568a in fetch_dlg_value (dlg=0x7f3b90508f78, name=name@entry=0x7f3bcf93c790, type=type@entry=0x7fffe6215a0c, out_val=out_val@entry=0x7fffe6215a10, val_has_buf=val_has_buf@entry=1)
    at dlg_vals.c:205
        dv = 0x7f3b90028740
        id = <optimized out>
        val = <optimized out>
        __FUNCTION__ = "fetch_dlg_value"
#1  0x00007f3b8ef0655c in pv_get_dlg_val (msg=0x7f3bcfbc70e8, param=0x7f3bcf93c780, res=0x7fffe6215cb8) at dlg_vals.c:310
        dlg = <optimized out>
        type = 1
        isval = {n = 21, s = {s = 0x7f3b00000015 <error: Cannot access memory at address 0x7f3b00000015>, len = 1}}
        __FUNCTION__ = "pv_get_dlg_val"
#2  0x0000563851012c92 in pv_get_spec_value (msg=msg@entry=0x7f3bcfbc70e8, sp=0x7f3bcf93c768, value=value@entry=0x7fffe6215cb8) at pvar.c:4830
        pv_msg = 0x7f3bcfbc70e8
        __FUNCTION__ = "pv_get_spec_value"
#3  0x0000563850fcda24 in get_cmd_fixups (msg=msg@entry=0x7f3bcfbc70e8, params=params@entry=0x7f3b8edbd9b0 <cmds+16>, elems=elems@entry=0x7f3bcf93c888, cmdp=cmdp@entry=0x7fffe6215c60,
    tmp_vals=tmp_vals@entry=0x7fffe6215ca0) at mod_fix.c:361
        i = 1
        param = 0x7f3b8edbd9b0 <cmds+16>
        gp = <optimized out>
        re = 0x0
        ret = <optimized out>
        __FUNCTION__ = "get_cmd_fixups"
#4  0x0000563850f70fe5 in do_action (a=a@entry=0x7f3bcf93c880, msg=msg@entry=0x7f3bcfbc70e8) at action.c:1041
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x5 <error: Cannot access memory at address 0x5>, len = -809733912}, ri = -812334608, flags = 32571}
        start = {tv_sec = 1728634922, tv_usec = 575521}
        end_time = <optimized out>
        cmd = 0x7f3b8edbd9a0 <cmds>
        acmd = <optimized out>
        cmdp = {0x5638512538d0 <__FUNCTION__.20>, 0x0, 0x7fffe6215e90, 0x7fffe6215d40, 0x7f3bcfaaceb0, 0x5638510dd630 <fm_malloc+323>, 0x7, 0x740000e164}
        tmp_vals = {{rs = {s = 0x0, len = 1358361700}, ri = -812334616, flags = 32571}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {
              s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = -811283648}, ri = 30, flags = 0}, {rs = {
              s = 0x5638510dd630 <fm_malloc+323> "I\215D$@H\203\304x[]A\\A]A^A_\303H\301\352\003\351\365\376\377\377\211\326H\301\346\004L\213d\036@M\205\344\017\205;\377\377\377\203\302\001\351\037\377\377\377\203\302\001A\211\320I\301\340\004M\213d\030@M\205\344\017\205\033\377\377\377\203\302\001A\211\321I\301\341\004M\213d\031@M\205\344\017\205\003\377\377\377\203\302\001A\211\322I\301\342\004M\213d\032@M\205\344\017\205\353\376\377\377\203\302\001A\211\323I\301\343\004M\213d\033@M\205\344\017\205\323\376\377\377\203\302\001A\211\324I\301\344\004M\213d\034@M\205\344\017\205\273\376\377\377\203\302\001\201\3723\b", len = -809733912}, ri = -809733912, flags = 32571}, {rs = {s = 0x6708e02a <error: Cannot access memory at address 0x6708e02a>, len = 575488}, ri = 1363356632, flags = 22072}, {rs = {
              s = 0x7f3bcfa4ca00 "!?\317;\177", len = 1728634922}, ri = 575489, flags = 0}, {rs = {s = 0x1 <error: Cannot access memory at address 0x1>, len = -1896856189}, ri = -809733912,
            flags = 32571}, {rs = {s = 0x563851322cc0 <_oser_err_info> "", len = 196611}, ri = 1, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#5  0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcf93c880
        ret = <optimized out>
        t = <optimized out>
#6  run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcf93c880) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#7  do_action (a=a@entry=0x7f3bcf94c710, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f3bcfa4d8d8 "\004", len = -809733912}, ri = 2, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575489}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcfbc9da0, 0x7fffe6215f70, 0x0, 0x2, 0x7f3bcf94c320, 0x5638510453d7 <eval_expr+1208>, 0x6, 0x56385101e164 <pv_value_destroy+81>}
        tmp_vals = {{rs = {s = 0x7f3bcfa4cd20 "\022", len = 1358361700}, ri = -811285136, flags = 32571}, {rs = {s = 0x892cc6d5c50d9e00 <error: Cannot access memory at address 0x892cc6d5c50d9e00>,
              len = 1361404729}, ri = -128, flags = -1}, {rs = {s = 0xb <error: Cannot access memory at address 0xb>, len = 1}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = -676626785,
            flags = 32571}, {rs = {s = 0x6708e02a <error: Cannot access memory at address 0x6708e02a>, len = 575475}, ri = 0, flags = 0}, {rs = {s = 0x56388c6e1300 "", len = 1728634922}, ri = 575475,
            flags = 0}, {rs = {s = 0x7fffe6216020 "", len = 0}, ri = -434020324, flags = 32767}, {rs = {s = 0x7fffe6216020 "", len = 0}, ri = -1938942960, flags = 22072}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#8  0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf94c710
#9  0x0000563850f723da in do_action (a=a@entry=0x7f3bcf94cb00, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f3bcfaa82b8 "\004", len = -809733912}, ri = 2, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575488}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x0, 0x563851044833 <dup_ref_script_route_in_shm+448>, 0x7fff00000080, 0x8, 0x7f3bcfbc9230, 0x56388c72de40, 0x56388c812ad0, 0x0}
        tmp_vals = {{rs = {s = 0x0, len = 0}, ri = 8, flags = 8}, {rs = {s = 0xb00000000 <error: Cannot access memory at address 0xb00000000>, len = 11}, ri = 11, flags = 6}, {rs = {s = 0x0,
              len = -1938942792}, ri = 16777473, flags = 0}, {rs = {s = 0x6 <error: Cannot access memory at address 0x6>, len = 1362248960}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0,
            flags = 0}, {rs = {s = 0x0, len = 1362248960}, ri = 0, flags = 0}, {rs = {s = 0x100000000 <error: Cannot access memory at address 0x100000000>, len = -811282472}, ri = -809733912,
            flags = 32571}, {rs = {s = 0x563851322cc0 <_oser_err_info> "", len = -810909000}, ri = 1, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x2 <error: Cannot access memory at address 0x2>, len = 0}
        __FUNCTION__ = "do_action"
#10 0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcf94cb00
        ret = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        t = <optimized out>
#11 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcf94cb00) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#12 do_action (a=a@entry=0x7f3bcfa4d8d8, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x0, len = -809733912}, ri = 0, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575488}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcfb714b8, 0x7fffe6216390, 0x7f3bcfbc70e8, 0x7fffe6216400, 0x7f3bcfa9f1e8, 0x12, 0x7f3bcfa9f5e8, 0x56385101e164 <pv_value_destroy+81>}
        tmp_vals = {{rs = {s = 0x7f3bcfa9f5e8 "\022", len = 1358361700}, ri = -434019456, flags = 32767}, {rs = {s = 0x7f3bcf929ed0 "", len = 0}, ri = 0, flags = 0}, {rs = {
              s = 0x7f3bcfbc8b30 "<sip:+492519811579812@3.65.209.120:5060;user=phone>", len = 51}, ri = 1361456069, flags = 22072}, {rs = {s = 0x7fffe62164b4 ";\177", len = -809722544}, ri = 11,
            flags = 0}, {rs = {s = 0x0, len = 0}, ri = -434019120, flags = 32767}, {rs = {s = 0x73 <error: Cannot access memory at address 0x73>, len = -809722544}, ri = 11, flags = 0}, {rs = {
              s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = -810922376}, ri = -809733912, flags = 32571}, {rs = {s = 0x563851322cc0 <_oser_err_info> "", len = 0}, ri = 4,
            flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#13 0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcfa4d8d8
        ret = <optimized out>
        t = <optimized out>
#14 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcfa4d8d8) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#15 do_action (a=a@entry=0x7f3bcfaa82b8, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        spec = <optimized out>
        val = {rs = {s = 0x0, len = 64}, ri = 2, flags = 32767}
        start = {tv_sec = 1728634922, tv_usec = 575473}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcfb58818, 0x7fff00000000, 0x7f3b00000000, 0xdbfa00, 0x563800000002, 0x7fff00000000, 0x0, 0x7f3b00000000}
        tmp_vals = {{rs = {s = 0xfffffffe <error: Cannot access memory at address 0xfffffffe>, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x7f3b8fe01030 "@>\340\217;\177", len = 128}, ri = -1881141200,
            flags = 32571}, {rs = {s = 0x80 <error: Cannot access memory at address 0x80>, len = -1886595432}, ri = 1360123897, flags = 22072}, {rs = {s = 0x7f3b8fe03640 "\001\001\002", len = 1360123897},
            ri = 1361504850, flags = 22072}, {rs = {s = 0x56385126eb80 <__FUNCTION__.1> "clone_sip_msg_body", len = 1359861296}, ri = -1, flags = 939524095}, {rs = {
              s = 0x892cc6d5c50d9e00 <error: Cannot access memory at address 0x892cc6d5c50d9e00>, len = 0}, ri = 160, flags = 0}, {rs = {
              s = 0x56385111d7f9 <_shm_malloc> "AWAVAUATUSH\203\354\030I\211\377I\211\364I\211\325A\211\316\2135\257R1", len = 1360123897}, ri = 0, flags = 0}, {rs = {s = 0x7f3b9144d188 "}",
              len = -809733912}, ri = 1360133801, flags = 22072}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#16 0x0000563850f74c60 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = <optimized out>
        t = 0x7f3bcfaa82b8
#17 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcfaa82b8) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#18 run_top_route (sr=..., msg=msg@entry=0x7f3bcfbc70e8) at action.c:250
        recursing = 1
        bk_action_flags = <optimized out>
        route_stack_start_bkp = <optimized out>
        route_stack_size_bkp = <optimized out>
        ret = <optimized out>
        ctx = 0x0
        __FUNCTION__ = "run_top_route"
#19 0x00007f3b8f188450 in pre_print_uac_request (body_clone=0x7fffe6216800, request=0x7f3bcfbc70e8, branch=0, t=0x7f3b8fe00fc0) at t_fwd.c:194
        backup_route_type = 1
        backup_list = 0x7f3b8fe02fe8
        p = <optimized out>
        error = <optimized out>
        backup_route_type = <optimized out>
        backup_list = <optimized out>
        p = <optimized out>
        __FUNCTION__ = "pre_print_uac_request"
        error = <optimized out>
#20 add_uac (t=t@entry=0x7f3b8fe00fc0, request=request@entry=0x7f3bcfbc70e8, uri=uri@entry=0x7fffe6216920, next_hop=next_hop@entry=0x7fffe6216910, bflags=<optimized out>, path=path@entry=0x7f3bcfbc7370,
    proxy=<optimized out>, proxy@entry=0x0) at t_fwd.c:427
        branch = 0
        body_clone = 0x7f3bcfbc9910
        do_free_proxy = <optimized out>
        ret = <optimized out>
        __FUNCTION__ = "add_uac"
#21 0x00007f3b8f18b086 in t_forward_nonack (t=t@entry=0x7f3b8fe00fc0, p_msg=p_msg@entry=0x7f3bcfbc70e8, proxy=proxy@entry=0x0, reset_bcounter=reset_bcounter@entry=0, locked=locked@entry=0) at t_fwd.c:770
        reply_reason_487 = {s = 0x7f3b8f1d1a3c "Request Terminated", len = 18}
        backup_uri = {s = 0x7f3bcfbc9330 "sip:++49199D4398492519811579812@3.65.209.120:5060;user=phone", len = 60}
--Type <RET> for more, q to quit, c to continue without paging--
        backup_dst = {s = 0x7f3bcfbc6e98 "sip:3.65.209.120:5060EPT=PCMA", len = 21}
        branch_ret = <optimized out>
        lowest_ret = -5
        current_uri = {s = 0x7f3bcfbc9330 "sip:++49199D4398492519811579812@3.65.209.120:5060;user=phone", len = 60}
        added_branches = 0
        i = <optimized out>
        q = 0
        t_invite = <optimized out>
        success_branch = <optimized out>
        dst_uri = {s = 0x0, len = 20}
        bk_sock = <optimized out>
        br_flags = 0
        bk_bflags = <optimized out>
        idx = <optimized out>
        path = {s = 0x0, len = 0}
        bk_path = <optimized out>
        __FUNCTION__ = "t_forward_nonack"
#22 0x00007f3b8f184d5b in t_relay_to (p_msg=p_msg@entry=0x7f3bcfbc70e8, proxy=proxy@entry=0x0, flags=flags@entry=0) at t_funcs.c:260
        ret = 0
        new_tran = 1
        reply_ret = <optimized out>
        t = 0x7f3b8fe00fc0
        __FUNCTION__ = "t_relay_to"
#23 0x00007f3b8f1a816c in w_t_relay (p_msg=0x7f3bcfbc70e8, flags=0x0, proxy=<optimized out>) at tm.c:1273
        p = 0x0
        t = 0x0
        ret = <optimized out>
        __FUNCTION__ = "w_t_relay"
#24 0x0000563850f7104f in do_action (a=a@entry=0x7f3bcfa49db0, msg=msg@entry=0x7f3bcfbc70e8) at action.c:1048
        ret = 0
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x56388c7da334 "8V", len = -809735417}, ri = -1895169536, flags = 32571}
        start = {tv_sec = 1728634922, tv_usec = 575351}
        end_time = <optimized out>
        cmd = 0x7f3b8f1e6370 <cmds+720>
        acmd = <optimized out>
        cmdp = {0x0, 0x0, 0x0, 0x7f3bcfbc70e8, 0x56385104511f <eval_expr+512>, 0x563851046142 <eval_expr+4643>, 0x3ff680000000000, 0x7f3bcfbc70e8}
        tmp_vals = {{rs = {s = 0x7fffe6216e90 "", len = 1607}, ri = 419, flags = 0}, {rs = {s = 0x7f3bcfbc70e8 "}", len = 35}, ri = 7, flags = 0}, {rs = {s = 0x563851239960 <__FUNCTION__.6> "del_lump",
              len = 1359861296}, ri = -434016884, flags = 32767}, {rs = {
              s = 0x5638510dd630 <fm_malloc+323> "I\215D$@H\203\304x[]A\\A]A^A_\303H\301\352\003\351\365\376\377\377\211\326H\301\346\004L\213d\036@M\205\344\017\205;\377\377\377\203\302\001\351\037\377\377\377\203\302\001A\211\320I\301\340\004M\213d\030@M\205\344\017\205\033\377\377\377\203\302\001A\211\321I\301\341\004M\213d\031@M\205\344\017\205\003\377\377\377\203\302\001A\211\322I\301\342\004M\213d\032@M\205\344\017\205\353\376\377\377\203\302\001A\211\323I\301\343\004M\213d\033@M\205\344\017\205\323\376\377\377\203\302\001A\211\324I\301\344\004M\213d\034@M\205\344\017\205\273\376\377\377\203\302\001\201\3723\b", len = 2}, ri = -676873763, flags = 32571}, {rs = {s = 0x7f3bd7bb6ae1 "<%d>%s%n%s%s%.0d%s: ", len = 0}, ri = 3432, flags = 0}, {rs = {
              s = 0x5638510dd630 <fm_malloc+323> "I\215D$@H\203\304x[]A\\A]A^A_\303H\301\352\003\351\365\376\377\377\211\326H\301\346\004L\213d\036@M\205\344\017\205;\377\377\377\203\302\001\351\037\377\377\377\203\302\001A\211\320I\301\340\004M\213d\030@M\205\344\017\205\033\377\377\377\203\302\001A\211\321I\301\341\004M\213d\031@M\205\344\017\205\003\377\377\377\203\302\001A\211\322I\301\342\004M\213d\032@M\205\344\017\205\353\376\377\377\203\302\001A\211\323I\301\343\004M\213d\033@M\205\344\017\205\323\376\377\377\203\302\001A\211\324I\301\344\004M\213d\034@M\205\344\017\205\273\376\377\377\203\302\001\201\3723\b", len = 30}, ri = -434016624, flags = 32767}, {rs = {s = 0x7f3bcfbc70e8 "}", len = 1359031442}, ri = 1359237407, flags = 22072}, {rs = {s = 0x0, len = 7}, ri = -1886595432, flags = 32571}}
        route_p = <optimized out>
        sval = {s = 0x3c <error: Cannot access memory at address 0x3c>, len = 1359861296}
        __FUNCTION__ = "do_action"
#25 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=msg@entry=0x7f3bcfbc70e8) at action.c:190
--Type <RET> for more, q to quit, c to continue without paging--
        ret = -1
        t = 0x7f3bcfa49db0
#26 0x00005638510453d7 in eval_elem (val=0x0, msg=0x7f3bcfbc70e8, e=0x7f3bcfa49ee8) at route.c:809
        ret = -5
        ival = <optimized out>
        i = <optimized out>
        lval = {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}
        rval = {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}
        p = <optimized out>
        n = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        ival = <optimized out>
        lval = <optimized out>
        rval = <optimized out>
        p = <optimized out>
        i = <optimized out>
        n = <optimized out>
        __FUNCTION__ = "eval_elem"
        error = <optimized out>
#27 eval_expr (e=0x7f3bcfa49ee8, msg=0x7f3bcfbc70e8, val=0x0) at route.c:1133
        rec_lev = 2
        ret = <optimized out>
        __FUNCTION__ = "eval_expr"
#28 0x0000563851045035 in eval_expr (e=0x7f3bcfa49f60, msg=msg@entry=0x7f3bcfbc70e8, val=val@entry=0x0) at route.c:1149
        rec_lev = 2
        ret = <optimized out>
        __FUNCTION__ = "eval_expr"
#29 0x0000563850f70230 in do_action (a=a@entry=0x7f3bcfa4b1a8, msg=msg@entry=0x7f3bcfbc70e8) at action.c:812
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f3bcfb8d1a8 "\320?\317;\177", len = -1881141776}, ri = -809971288, flags = 32571}
        start = {tv_sec = 1728634922, tv_usec = 575351}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcf931d80, 0x9, 0x7f3bcfbc70e8, 0x7f3bcf931eb8, 0x7f3bcfbc70e8, 0x7fffe6217220, 0x7f3bcf931cd8, 0x12}
        tmp_vals = {{rs = {s = 0x7f3bcf931eb8 "\022", len = 1358361700}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0,
              len = 0}, ri = -1873768584, flags = 32571}, {rs = {s = 0x7f3bcfbc70e8 "}", len = 0}, ri = -1897782432, flags = 32571}, {rs = {s = 0x0, len = -434015376}, ri = -1897844275, flags = 32571}, {
            rs = {s = 0x7f3bcfbc70e8 "}", len = -809971288}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x0, len = 0}
        __FUNCTION__ = "do_action"
#30 0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcfa4b1a8
        ret = <optimized out>
        t = <optimized out>
#31 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcfa4b1a8) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#32 do_action (a=a@entry=0x7f3bcf9320b0, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x0, len = 1}, ri = 0, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575348}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcfb8d1a8, 0x7fffe62173e0, 0x0, 0x7f3bcfbc70e8, 0x1, 0x563851044fe2 <eval_expr+195>, 0x7f3bcfbc9230, 0x0}
        tmp_vals = {{rs = {s = 0x0, len = -676287701}, ri = 0, flags = 0}, {rs = {s = 0x7f3bcf92e340 "outroute", len = 8}, ri = -676287765, flags = 32571}, {rs = {s = 0x7f3bcfb49220 "2LOEWEN_GW5",
              len = 11}, ri = -1878889864, flags = 32571}, {rs = {s = 0x7fffe6217550 "\230?\220;\177", len = -809973896}, ri = 104, flags = 0}, {rs = {
              s = 0xb <error: Cannot access memory at address 0xb>, len = 116}, ri = 11, flags = 0}, {rs = {s = 0x5638510402b2 <comp_s2s+3401> "\205\300\017\224\303\017\266\303\351\375\362\377\377L\213e",
              len = -675583262}, ri = 0, flags = 7}, {rs = {s = 0xb00000000 <error: Cannot access memory at address 0xb00000000>, len = -434015056}, ri = 808530191, flags = 808530489}, {rs = {
              s = 0x892cc6d5c50d9e00 <error: Cannot access memory at address 0x892cc6d5c50d9e00>, len = -434014192}, ri = -434014192, flags = 32767}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#33 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf9320b0
#34 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf9321e8, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575325}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x1, 0x7f3bcf92cf00, 0x7f3bcf914330, 0x7f3bcfbc70e8, 0x1, 0x563851044fe2 <eval_expr+195>, 0x7fffe6217800, 0x7f3bcfbc70e8}
        tmp_vals = {{rs = {s = 0x0, len = -1896468046}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = -676287765, flags = 32571}, {rs = {s = 0x7fffe62177f0 "", len = 0}, ri = -1878889864,
            flags = 32571}, {rs = {s = 0x7fffe62177b0 "\230?\220;\177", len = -809974408}, ri = -812465936, flags = 32571}, {rs = {s = 0x14 <error: Cannot access memory at address 0x14>,
              len = -434014208}, ri = -809733912, flags = 32571}, {rs = {s = 0x5638510409d7 <comp_s2s+5230> "L\213\030M\211\351I\367\321I\001\351A\203\341\aM\215w\001I\017\276}", len = -1937696130},
            ri = -128, flags = -1}, {rs = {s = 0x7f3bcfb8bf80 "P-Germany-Tariff", len = 16}, ri = 0, flags = 4}, {rs = {s = 0xb <error: Cannot access memory at address 0xb>, len = 0}, ri = -1894838816,
            flags = 32571}}
        route_p = <optimized out>
        sval = {s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = 0}
        __FUNCTION__ = "do_action"
#35 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
--Type <RET> for more, q to quit, c to continue without paging--
        ret = -1
        t = 0x7f3bcf9321e8
#36 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf932320, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = -988963328}, ri = -434014128, flags = 1}
        start = {tv_sec = 1728634922, tv_usec = 575324}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x1, 0x7f3bcf92c210, 0x1, 0x7f3bcfbc70e8, 0x1, 0x563851044fd4 <eval_expr+181>, 0x7f3bcfb8b058, 0x0}
        tmp_vals = {{rs = {s = 0x7fffe6217bb0 "0C\221\317;\177", len = -1871014008}, ri = -1896761338, flags = 32571}, {rs = {
              s = 0x892cc6d5c50d9e00 <error: Cannot access memory at address 0x892cc6d5c50d9e00>, len = 109}, ri = -434013424, flags = 32767}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0,
              len = 0}, ri = -676337571, flags = 32571}, {rs = {s = 0x56385104511f <eval_expr+512> "A\213l$\004\203\375\037\017\204\345\a", len = 0}, ri = -1871015256, flags = 32571}, {rs = {
              s = 0x892cc6d5c50d9e00 <error: Cannot access memory at address 0x892cc6d5c50d9e00>, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x7fffe6217b60 "`\264\222\317;\177", len = -809733912},
            ri = -434013344, flags = 32767}, {rs = {s = 0x7f3bcf92a6b8 "", len = 88}, ri = 1359055912, flags = 22072}}
        route_p = <optimized out>
        sval = {s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = 1360727536}
        __FUNCTION__ = "do_action"
#37 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf932320
#38 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf932458, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x19 <error: Cannot access memory at address 0x19>, len = -812472392}, ri = 1, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575309}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x14, 0x7f3bcf92b520, 0x0, 0x7f3bcfbc70e8, 0x0, 0x563851045e1d <eval_expr+3838>, 0x27fff00000003, 0x7fffe6217ba0}
        tmp_vals = {{rs = {s = 0x7f3bcfbc70e8 "}", len = 31648}, ri = 0, flags = 0}, {rs = {s = 0x12 <error: Cannot access memory at address 0x12>, len = -1873769064}, ri = 3, flags = 0}, {rs = {
              s = 0x7f3bcf925800 "\022", len = 1358361700}, ri = -809725392, flags = 32571}, {rs = {s = 0x7f3bcf92aad0 "\004", len = 20}, ri = -434012992, flags = 32767}, {rs = {s = 0x7f3bcfbc70e8 "}",
              len = -809733912}, ri = 0, flags = 0}, {rs = {s = 0x563851012c92 <pv_get_spec_value+159> "\205\300\017\205", <incomplete sequence \307>, len = -809734712}, ri = 4, flags = 0}, {rs = {
              s = 0x7f3bcf92b460 "false", len = 5}, ri = 4393, flags = 28}, {rs = {s = 0x6 <error: Cannot access memory at address 0x6>, len = -809734712}, ri = 4, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x7fffe6217cc0 "\230\215P\220;\177", len = -812471576}
        __FUNCTION__ = "do_action"
#39 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf932458
#40 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf932590, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 0
        i = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = 1358980531}, ri = 2, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575309}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x1, 0x7f3bcf92abf0, 0x1, 0x7f3bcfbc70e8, 0x1, 0x563851044fd4 <eval_expr+181>, 0x1, 0x16}
        tmp_vals = {{rs = {s = 0x7f3bcf925bd8 "true", len = 4}, ri = -812491504, flags = 32571}, {rs = {s = 0x7f3bcfbc70e8 "}", len = -434012672}, ri = -812491984, flags = 32571}, {rs = {
              s = 0x7f3bcf925d10 "\022", len = 1358361700}, ri = 9, flags = 124}, {rs = {s = 0x1 <error: Cannot access memory at address 0x1>, len = 1109210}, ri = -434012680, flags = 32767}, {rs = {
              s = 0x5638512a48ec "", len = 123}, ri = -810353976, flags = 32571}, {rs = {
              s = 0x5638510d3b36 <xlog_2+300> "\351\374\376\377\377H\215T$PH\203\306\bH\211|$8\350\245\360\363\377\205\300\017\205 \004", len = -1871015056}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0},
            ri = 123, flags = 0}, {rs = {s = 0x7f3bcfb2fac8 "++++++++ avp(gw_attrs) = DPO=4394;EO=9000;ROUTE=2LOEWEN_GW5;CAPS=10;NPDI=NO;CCC=60;RG=4393;PK_REM=NO;PI=ID;ANO=+49251981150",
              len = -1871015056}, ri = 0, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x400000000 <error: Cannot access memory at address 0x400000000>, len = -434012672}
        __FUNCTION__ = "do_action"
#41 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf932590
#42 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf9326c8, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 0
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x1 <error: Cannot access memory at address 0x1>, len = -809733912}, ri = 1, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575307}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcf912d80, 0x7f3b90fa0c90, 0x56385101ce39 <pv_printf+493>, 0x563851322b40 <ctime_buf>, 0x563851322b40 <ctime_buf>, 0x5638510dd630 <fm_malloc+323>, 0x51322b59, 0x7fffe6217fd0}
        tmp_vals = {{rs = {s = 0x563851322b59 <ctime_buf+25> "", len = 0}, ri = -812568848, flags = 32571}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 57}, {rs = {s = 0x0, len = -434011904},
            ri = -1871016824, flags = 32571}, {rs = {s = 0x101820000000f <error: Cannot access memory at address 0x101820000000f>, len = -809733912}, ri = 1359031442, flags = 22072}, {rs = {
              s = 0x6708e02a <error: Cannot access memory at address 0x6708e02a>, len = 575185}, ri = 2, flags = 0}, {rs = {s = 0x7f3b8f8cd698 "", len = 1728634922}, ri = 575207, flags = 0}, {rs = {
              s = 0x54 <error: Cannot access memory at address 0x54>, len = 1359783132}, ri = 0, flags = 0}, {rs = {s = 0x563851018c28 <pv_get_avp+161> "H\211\303H\205\300\017\204g\002",
              len = -434011992}, ri = 2, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x7fffe6218040 "\250\200!\346\377\177", len = 0}
        __FUNCTION__ = "do_action"
#43 0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcf9326c8
        ret = <optimized out>
        t = <optimized out>
#44 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcf9326c8) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#45 do_action (a=a@entry=0x7f3bcf914330, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x7f3bcf8bed90 "\004", len = -809733912}, ri = 2, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575209}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3b907a8c88, 0x101820000000f, 0x0, 0x2, 0x7f3bcf913028, 0x5638510453d7 <eval_expr+1208>, 0xa, 0x7fffe6218290}
        tmp_vals = {{rs = {s = 0x7f3bcfbc70e8 "}", len = -434011504}, ri = -812572424, flags = 32571}, {rs = {s = 0x12 <error: Cannot access memory at address 0x12>, len = 2}, ri = 1359077732,
            flags = 22072}, {rs = {s = 0x7f3bcf912448 "\022", len = 1358361700}, ri = 9, flags = 124}, {rs = {s = 0x1 <error: Cannot access memory at address 0x1>, len = 1109210}, ri = -434011512,
            flags = 32767}, {rs = {s = 0x5638512a48ec "", len = 51}, ri = -810353976, flags = 32571}, {rs = {
              s = 0x5638510d3b36 <xlog_2+300> "\351\374\376\377\377H\215T$PH\203\306\bH\211|$8\350\245\360\363\377\205\300\017\205 \004", len = 1361360431}, ri = 1362242372, flags = 22072}, {rs = {
              s = 0x7f3b0010ecda <error: Cannot access memory at address 0x7f3b0010ecda>, len = 1361725676}, ri = 51, flags = 22072}, {rs = {
              s = 0x7f3bcfb2fac8 "++++++++ avp(gw_attrs) = DPO=4394;EO=9000;ROUTE=2LOEWEN_GW5;CAPS=10;NPDI=NO;CCC=60;RG=4393;PK_REM=NO;PI=ID;ANO=+49251981150", len = 1363338869}, ri = 4, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#46 0x0000563850f748c1 in run_action_list (a=<optimized out>, msg=0x7f3bcfbc70e8) at action.c:190
        ret = -1
        t = 0x7f3bcf914330
#47 0x0000563850f723da in do_action (a=a@entry=0x7f3bcf914468, msg=msg@entry=0x7f3bcfbc70e8) at action.c:836
        ret = 1
        v = 1
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0xffffffff <error: Cannot access memory at address 0xffffffff>, len = -809733912}, ri = 1, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575185}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7f3bcfbc6af0, 0x7fffe62184c0, 0x7f3bcfbc70e8, 0x7fffe62184c0, 0x7f3bcf8bcf60, 0x12, 0x7f3bcf8bd1e8, 0x56385101e164 <pv_value_destroy+81>}
        tmp_vals = {{rs = {s = 0x7f3bcf8bd1e8 "\022", len = 1358361700}, ri = -813650120, flags = 32571}, {rs = {s = 0x7f3bcf831e10 "inroute", len = 7}, ri = -434010496, flags = 32767}, {rs = {
              s = 0x7f3bcfbc8ba8 "<sip:+4936439062875;npdi@telekom.de;user=phone?Privacy=history>;index=1\r\n", len = 394}, ri = 0, flags = 0}, {rs = {
              s = 0x7f3b00000000 <error: Cannot access memory at address 0x7f3b00000000>, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 0,
            flags = 0}, {rs = {s = 0x100000000 <error: Cannot access memory at address 0x100000000>, len = -812921808}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = 3, flags = 0}}
        route_p = <optimized out>
        sval = {s = 0x2 <error: Cannot access memory at address 0x2>, len = 0}
        __FUNCTION__ = "do_action"
#48 0x0000563850f72254 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = -1
        t = 0x7f3bcf914468
        ret = <optimized out>
--Type <RET> for more, q to quit, c to continue without paging--
        t = <optimized out>
#49 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcf914468) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#50 do_action (a=a@entry=0x7f3bcf8bed90, msg=msg@entry=0x7f3bcfbc70e8) at action.c:803
        ret = -5
        v = <optimized out>
        i = <optimized out>
        len = <optimized out>
        cmatch = <optimized out>
        aitem = <optimized out>
        adefault = <optimized out>
        spec = <optimized out>
        val = {rs = {s = 0x0, len = 1359861296}, ri = 0, flags = 0}
        start = {tv_sec = 1728634922, tv_usec = 575171}
        end_time = <optimized out>
        cmd = 0x0
        acmd = <optimized out>
        cmdp = {0x7fffe6218678, 0x7fffe6218680, 0x7f3bcf86bc98, 0x0, 0x0, 0x0, 0x0, 0x0}
        tmp_vals = {{rs = {s = 0x0, len = 0}, ri = 0, flags = 495}, {rs = {s = 0x7f3bcf86ba58 "inroute", len = 7}, ri = 100, flags = 32571}, {rs = {s = 0x7f3bcf889740 "Giving a try", len = 12}, ri = -1,
            flags = -1}, {rs = {
              s = 0x56385126dad4 "B'\352\377\322%\352\377\247'\352\3771/\352\377\303(\352\377\333(\352\377X)\352\377\016)\352\377\263%\352\377\363(\352\377p)\352\377&)\352\377\267)\352\377\351)\352\377\272,\352\377F+\352\377\322-\352\377\240*\352\377C-\352\377\024,\352\377t.\352\3776*\352\377\360,\352\377\252+\352\377\n.\352\377\363*\352\377\226-\352\377g,\352\377\336.\352\377\004*\352\377\325,\352\377x+\352\377\355-\352\377\322*\352\377u-\352\377F,\352\377\246.\352\377h*\352\377\"-\352\377\322%\352\377\334+\352\377<.\352\377\016+\352\377\261-\352\377\202,\352\377\371.\352\377\322%\352\377\322%\352\377\322%\352\377\322%\352\377"..., len = 1363498422}, ri = 1360069137, flags = 22072}, {rs = {s = 0x0, len = 0}, ri = 0, flags = 0}, {rs = {s = 0x0, len = 0}, ri = -1862616440, flags = 32571}, {rs = {
              s = 0x7f3b8f1d81ec "t_reply.c", len = -1893885808}, ri = 174, flags = 0}, {rs = {s = 0x563851432b58 <shm_block> "", len = 72}, ri = -1894019672, flags = 32571}}
        route_p = <optimized out>
        sval = {s = 0x5638512301bf "unknown", len = 7}
        __FUNCTION__ = "do_action"
#51 0x0000563850f74c60 in run_action_list (msg=<optimized out>, a=<optimized out>) at action.c:190
        ret = <optimized out>
        t = 0x7f3bcf8bed90
#52 run_actions (msg=0x7f3bcfbc70e8, a=0x7f3bcf8bed90) at action.c:136
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        error = <optimized out>
        ret = <optimized out>
        _ = <optimized out>
        top_route = <optimized out>
        __FUNCTION__ = "run_actions"
        error = <optimized out>
#53 run_top_route (sr=..., msg=msg@entry=0x7f3bcfbc70e8) at action.c:250
        recursing = 1
        bk_action_flags = <optimized out>
        route_stack_start_bkp = <optimized out>
        route_stack_size_bkp = <optimized out>
        ret = <optimized out>
        ctx = 0x0
        __FUNCTION__ = "run_top_route"
--Type <RET> for more, q to quit, c to continue without paging--
#54 0x000056385102c0b6 in receive_msg (
    buf=0x563851455160 <buf> "INVITE sip:+49199D4398492519811579812@10.249.152.9:5060;user=phone SIP/2.0\r\nVia: SIP/2.0/UDP 10.249.152.5:5060;branch=z9hG4bKd498.dd854d76.0\r\nFrom: <sip:+4917672856327@telefonica-voip.de;user=phone>;t"..., len=<optimized out>, rcv_info=rcv_info@entry=0x7fffe6218990, existing_context=existing_context@entry=0x0, msg_flags=msg_flags@entry=0) at receive.c:225
        ctx = 0x7f3bcfbc3a58
        msg = 0x7f3bcfbc70e8
        start = {tv_sec = 1728634922, tv_usec = 573832}
        rc = 3
        old_route_type = <optimized out>
        tmp = <optimized out>
        in_buff = {
          s = 0x563851455160 <buf> "INVITE sip:+49199D4398492519811579812@10.249.152.9:5060;user=phone SIP/2.0\r\nVia: SIP/2.0/UDP 10.249.152.5:5060;branch=z9hG4bKd498.dd854d76.0\r\nFrom: <sip:+4917672856327@telefonica-voip.de;user=phone>;t"..., len = 1607}
        __FUNCTION__ = "receive_msg"
#55 0x0000563851217c4a in udp_read_req (si=<optimized out>, bytes_read=<optimized out>) at net/proto_udp/proto_udp.c:186
        ri = {src_ip = {af = 2, len = 4, u = {addrl = {93911306, 0}, addr32 = {93911306, 0, 0, 0}, addr16 = {63754, 1432, 0, 0, 0, 0, 0, 0}, addr = "\n\371\230\005", '\000' <repeats 11 times>}}, dst_ip = {
            af = 2, len = 4, u = {addrl = {161020170, 0}, addr32 = {161020170, 0, 0, 0}, addr16 = {63754, 2456, 0, 0, 0, 0, 0, 0}, addr = "\n\371\230\t", '\000' <repeats 11 times>}}, src_port = 5060,
          dst_port = 5060, proto = 1, proto_reserved1 = 0, proto_reserved2 = 0, src_su = {s = {sa_family = 2, sa_data = "\023\304\n\371\230\005\000\000\000\000\000\000\000"}, sin = {sin_family = 2,
              sin_port = 50195, sin_addr = {s_addr = 93911306}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = {sin6_family = 2, sin6_port = 50195, sin6_flowinfo = 93911306, sin6_addr = {__in6_u = {
                  __u6_addr8 = "\000\000\000\000\000\000\000\000O\000\000\000\000\000\000", __u6_addr16 = {0, 0, 0, 0, 79, 0, 0, 0}, __u6_addr32 = {0, 0, 79, 0}}}, sin6_scope_id = 3862927285}},
          bind_address = 0x7f3bcf80af38}
        len = <optimized out>
        buf = "INVITE sip:+49199D4398492519811579812@10.249.152.9:5060;user=phone SIP/2.0\r\nVia: SIP/2.0/UDP 10.249.152.5:5060;branch=z9hG4bKd498.dd854d76.0\r\nFrom: <sip:+4917672856327@telefonica-voip.de;user=phone>;t"...
        tmp = <optimized out>
        fromlen = 16
        p = <optimized out>
        msg = {
          s = 0x563851455160 <buf> "INVITE sip:+49199D4398492519811579812@10.249.152.9:5060;user=phone SIP/2.0\r\nVia: SIP/2.0/UDP 10.249.152.5:5060;branch=z9hG4bKd498.dd854d76.0\r\nFrom: <sip:+4917672856327@telefonica-voip.de;user=phone>;t"..., len = 1607}
        __FUNCTION__ = "udp_read_req"
#56 0x00005638511ffb73 in handle_io (idx=<optimized out>, event_type=<optimized out>, fm=<optimized out>) at net/net_udp.c:295
        n = <optimized out>
        read = 0
        n = <optimized out>
        read = <optimized out>
        __FUNCTION__ = "handle_io"
#57 io_wait_loop_epoll (repeat=<optimized out>, t=<optimized out>, h=<optimized out>) at net/../io_wait_loop.h:311
        ret = <optimized out>
        n = <optimized out>
        r = 0
        i = <optimized out>
        e = <optimized out>
        ep_event = {events = 0, data = {ptr = 0xd7af2d9300000000, fd = 0, u32 = 0, u64 = 15541690948462313472}}
        fd = <optimized out>
        curr_time = 11
        error = <optimized out>
        __FUNCTION__ = "io_wait_loop_epoll"
#58 0x00005638512051db in udp_start_processes (chd_rank=chd_rank@entry=0x5638513237b8 <chd_rank>, startup_done=startup_done@entry=0x7f3b910fd228) at net/net_udp.c:528
        si = <optimized out>
        p_id = <optimized out>
        i = <optimized out>
        p = <optimized out>
        ifp_udp_rcv = {proc_desc = 0x5638512b5f8f "UDP receiver", flags = 4, type = TYPE_UDP}
        __FUNCTION__ = "udp_start_processes"
        error = <optimized out>
#59 0x0000563850f6c4be in main_loop () at main.c:237
        startup_done = 0x7f3b910fd228
--Type <RET> for more, q to quit, c to continue without paging--
        last_check = 0
        rc = <optimized out>
        chd_rank = 31
        startup_done = <optimized out>
        last_check = <optimized out>
        rc = <optimized out>
        profiling_handler = {desc = 0x5638512407ac "_ProfilerStart_child()", on_child_init = 0x563850fc6a80 <_ProfilerStart_child>, _next = 0x0}
        __FUNCTION__ = "main_loop"
        error = <optimized out>
#60 main (argc=<optimized out>, argv=<optimized out>) at main.c:966
        c = <optimized out>
        r = <optimized out>
        tmp = 0x7fffe621ad7c ""
        tmp_len = <optimized out>
        port = <optimized out>
        proto = <optimized out>
        protos_no = <optimized out>
        options = 0x56385123e358 "f:cCm:M:b:l:n:N:rRvdDFEVhw:t:u:g:p:P:G:W:o:a:k:s:"
        ret = -1
        seed = 1356845333
        rfd = <optimized out>
        procs_no = <optimized out>
        __FUNCTION__ = "main"
(gdb)
@bogdan-iancu
Copy link
Member

Hi @MarcoHierl, could you first re-compile without optimization (so we can get better gdb info). In Makefile.conf, be sure this is uncommented:

DEFS+= -DCC_O0 #Zero compiler optimizations (FAST compile, SLOW code. For devs)

Make it crash again and post the full backtrace as first. Thanks!

@bogdan-iancu bogdan-iancu self-assigned this Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bogdan-iancu bogdan-iancu

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bogdan-iancu @MarcoHierl