| 00:14.42 | *** part/#asterisk-dev kharwell (kharwell@nat/digium/x-sztfxvxmrgduxvcr) |
| 00:41.24 | gtjoseph | file so on the alembic conflict, this is master only correct? 13 is OK? |
| 00:41.38 | file | Mark's 13 change went in awhile ago |
| 00:41.44 | gtjoseph | gotcha |
| 00:47.15 | gtjoseph | file: no issues running 'branches' and 'history looks clean. |
| 00:47.30 | file | yay? |
| 00:47.37 | gtjoseph | yeah, marks commits went into 13 |
| 00:47.52 | gtjoseph | same hashs as master |
| 00:48.49 | gtjoseph | something went right for a change |
| 00:51.02 | file | ok, this is concerning |
| 00:51.07 | file | let's seeeeeee |
| 00:51.09 | gtjoseph | why? |
| 00:51.36 | file | unit test execution failure |
| 00:51.40 | file | need to look at the Jenkins node |
| 00:52.32 | gtjoseph | oh, websockets in 13 again. |
| 00:52.50 | file | in particular that agent |
| 00:53.21 | file | there - should be fixed |
| 00:53.33 | file | the test that was stuck however... |
| 00:53.41 | file | was tests/pbx/dialplan_reload |
| 00:55.30 | coreyfarrell | speaking of websockets, any objection to making it not an optional_api in master? the only module that 'optionally' uses it is chan_sip and that has a config option now. |
| 00:56.07 | gtjoseph | wfm fwiw |
| 00:56.43 | file | ummm depends on when symbol resolution is done... you'll make it a hard dependency in chan_sip |
| 00:57.43 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 00:58.56 | coreyfarrell | file: yes it will become required to load chan_sip, but the functionality can be disabled in config. the symbol resolution would enforce load order so res_http_websocket cannot be loaded after chan_sip, res_ari or res_pjsip_transport_websocket |
| 00:59.37 | file | required to load chan_sip? I meant it'll be required to have res_http_websocket loaded on some systems, depending on if the symbols are resolved at load time or lazy when used |
| 01:00.56 | coreyfarrell | I thought the module loader did lazy first then reopened with immediate resolves..? |
| 01:01.02 | coreyfarrell | looks at code |
| 01:03.32 | file | I think the asterisk-dev mailing list would be better for such a discussion regardless |
| 01:04.15 | coreyfarrell | ok so looks like the final dlopen uses RTLD_LAZY for modules that export globals, RTLD_NOW otherwise. |
| 01:05.20 | coreyfarrell | file: yeah it's not as straight forward as I was thinking anyways |
| 01:10.08 | file | needs more fire |
| 01:18.25 | *** join/#asterisk-dev gtjoseph (~gtj0@unaffiliated/gtj) |
| 03:09.37 | *** join/#asterisk-dev leedm777 (~leedm777@50-81-28-152.client.mchsi.com) |
| 03:31.33 | *** join/#asterisk-dev leedm777 (~leedm777@50-81-28-152.client.mchsi.com) |
| 04:43.51 | *** join/#asterisk-dev leedm777 (~leedm777@50-81-28-152.client.mchsi.com) |
| 07:00.09 | *** join/#asterisk-dev bof22 (~Thunderbi@185.13.183.107) |
| 07:31.20 | *** join/#asterisk-dev hehol (~hehol@gatekeeper.loca.net) |
| 07:33.35 | *** join/#asterisk-dev Demon_VoIP (~demon@109.60.135.186) |
| 08:12.16 | *** join/#asterisk-dev opticron (~opticron@75.76.201.152) |
| 08:12.16 | *** mode/#asterisk-dev [+o opticron] by ChanServ |
| 08:17.22 | *** join/#asterisk-dev tzafrir (~tzafrir@local.xorcom.com) |
| 08:17.22 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 08:22.21 | *** join/#asterisk-dev pchero_work (~pchero@109.70.54.56) |
| 08:55.59 | *** join/#asterisk-dev hehol (~hehol@gatekeeper.loca.net) |
| 09:20.50 | *** join/#asterisk-dev goddva (~goddva@77.40.154.242) |
| 10:19.36 | *** join/#asterisk-dev bof23 (~Thunderbi@185.13.183.107) |
| 11:53.57 | *** join/#asterisk-dev lnxslck (~lnxslck@av3-84-90-160-135.netvisao.pt) |
| 11:56.30 | Demon_VoIP | gtjoseph: Have you inspected the file with the lock in __ast_bt_get_addresses from qualify_contact? Today asterisk hanged again with this lock :( |
| 12:31.35 | *** join/#asterisk-dev bof22 (~Thunderbi@185.13.183.107) |
| 13:22.09 | *** join/#asterisk-dev [TK]D-Fender (~chatzilla@216-191-106-163.dedicated.allstream.net) |
| 13:24.54 | *** join/#asterisk-dev lnxslck (~lnxslck@av3-84-90-160-135.netvisao.pt) |
| 13:45.05 | file | Tuesday is. |
| 13:46.34 | sgriepentrog | You forgot "it" |
| 13:49.44 | *** join/#asterisk-dev leedm777 (leedm777@nat/digium/x-qtfssanupljqsvxs) |
| 14:42.13 | gtjoseph | Demon_VoIP: I haven't yet. I was looking at the segfault first. I'll look now. |
| 14:54.20 | gtjoseph | file: we do have another test that binds to something other than 127.0.0.1: configuration/duplicate_sections |
| 14:55.45 | gtjoseph | and i'm not sure how to test binding if i can't bind to something other than 127.0.0.1 |
| 14:55.48 | file | it doesn't actually pass traffic |
| 14:56.07 | file | I'm fairly certain it was you could bind, but the source would be 127.0.0.1 even if sent over 127.0.0.2 |
| 14:56.13 | file | and I think it was chan_iax2 that did it... |
| 14:56.19 | gtjoseph | hmmm |
| 14:56.23 | file | and some PJSIP tests, which were changed to use a port instead |
| 14:59.57 | gtjoseph | file: iax2/acl_call binds to 127.0.0.[1-4] |
| 15:00.52 | file | maybe it'll work then... but I do remember this being a problem |
| 15:01.31 | file | we'll see! |
| 15:01.52 | gtjoseph | i can't find another option unfortunately |
| 15:04.06 | file | aye |
| 15:06.25 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 15:11.22 | *** join/#asterisk-dev kharwell (kharwell@nat/digium/x-pjhnpuhxaforxpaj) |
| 15:13.58 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 15:40.48 | *** join/#asterisk-dev putnopvut (putnopvut@asterisk/master-of-queues/mmichelson) |
| 15:40.48 | *** mode/#asterisk-dev [+o putnopvut] by ChanServ |
| 15:42.20 | putnopvut | gtjoseph: Got an interesting problem: https://issues.asterisk.org/jira/browse/ASTERISK-25689 |
| 15:42.36 | gtjoseph | is looking |
| 15:42.37 | putnopvut | Looks like commit 162acd45 resulted in this |
| 15:42.50 | putnopvut | (Adding "like" support for CLI commands) |
| 15:43.16 | putnopvut | The problem is that sending an empty string regex to res_sorcery_realtime results in res_sorcery_realtime returning an empty list of objects. |
| 15:43.36 | gtjoseph | hmmm |
| 15:43.38 | putnopvut | It's because res_sorcery_realtime checks the regex string to see if the first char is "^". If it's not, then it returns immediately. |
| 15:43.46 | putnopvut | So there are a few ways we can go to fix this. |
| 15:43.47 | gtjoseph | Ha! |
| 15:43.57 | putnopvut | 1. Remove that check from res_sorcery_realtime |
| 15:44.12 | putnopvut | 2. Send a different regex to sorcery backends than an empty string |
| 15:44.18 | putnopvut | 3. ...something else |
| 15:44.41 | putnopvut | I think (1) sounds best to me, becuase I don't really understand why that check is present in res_sorcery_realtime in the first place. |
| 15:44.50 | gtjoseph | what do you think the 'correct' behavior should be for the backends? |
| 15:45.00 | gtjoseph | yeah, i'd agree |
| 15:45.19 | gtjoseph | whichever the solution, the backends should all behave thew same |
| 15:45.30 | file | I'm sure I did it for a reason, but I can't remember why |
| 15:45.49 | putnopvut | Correct, different behaviors from backends is odd. |
| 15:46.05 | file | oh |
| 15:46.27 | putnopvut | Now in my head, whenever I write a "match all" regex, I'd write something like "^.*$" or just ".*" but the problem is that realtime doesn't actually have regex support, so those wouldn't match everything like you might expect. |
| 15:46.28 | gtjoseph | requiring the ^ would seem to limit the match to beginning of string only |
| 15:46.57 | file | yeah, it's because it's a layer built on realtime instead of directly |
| 15:48.21 | putnopvut | So do you guys agree that altering res_sorcery_realtime not to require a leading "^" is the best way to fix this? |
| 15:48.30 | gtjoseph | i do |
| 15:48.33 | putnopvut | I believe that would make all the sorcery backends behave the same then. |
| 15:48.53 | file | well, potentially, depends on what the underlying database does with the value provided |
| 15:49.12 | file | (from a behaving the same) |
| 15:49.29 | file | but yeah, remove it and see if it works as expected :D |
| 15:49.35 | gtjoseph | well, it's processed as a LIKE |
| 15:49.49 | gtjoseph | so i'm assuming it needs SQL syntax |
| 15:50.46 | gtjoseph | hmmm |
| 15:51.26 | gtjoseph | if you mod realtime to not need the ^ you should also change the "%s%%" to "%%%s%%" |
| 15:56.23 | putnopvut | Doing an initial compile of 13 right now, will test the change to see if that works, but I also have a meeting in 5 minutes, so it probably won't be until after then that I'd be able to confirm the change. |
| 16:01.21 | *** join/#asterisk-dev rmudgett (rmudgett@nat/digium/x-qqnktehrnmgbsudk) |
| 16:20.51 | *** join/#asterisk-dev Bordr (~Bordr@23-24-156-49-static.hfc.comcastbusiness.net) |
| 16:36.01 | putnopvut | Looks like that change worked fine. I'll put it up for review. |
| 16:44.59 | gtjoseph | hmmm. how come we don't store time locked in ast_lock_track. Seems like it would be useful. |
| 16:46.50 | stefan27 | regarding ASTERISK-25642 , I made a patch which adds an additional ast_srtp field member to each ast_rtp_instance , which I try manipulate wherever asterisk manipulates the other (the old) ast_srtp field (with small modifications). I have tested the patch today and it seems to solve the big issue with dtls-encrypted video, but it's really brute so should I bother cleaning it up for gerrit |
| 16:46.50 | stefan27 | review or is the solution too messy? |
| 16:47.28 | stefan27 | patch-code that probably will never go to asterisk-trunk should just be an attachment to the jira? |
| 16:48.48 | file | stefan27, you could - but be sure to test SDES as well... |
| 16:49.25 | stefan27 | I wrote it so that when there is no dtls-session on the rtp instance in question behaviour will be as before |
| 16:49.40 | file | doesn't mean something didn't sneak in, it happens :D |
| 16:50.03 | *** join/#asterisk-dev roramirez (~rodrigo@181.75.126.119) |
| 16:50.13 | roramirez | Hello |
| 16:54.45 | file | putnopvut, you haz broke unit test |
| 16:54.46 | stefan27 | it's a shame the senior c-dev in our company only works 10% now that he's on parental leave |
| 16:54.58 | putnopvut | boo |
| 16:55.11 | file | which specifically tests regex support - lawlz |
| 16:55.19 | stefan27 | im gonna run some more tests |
| 16:55.32 | putnopvut | file: which one? |
| 16:55.41 | file | https://jenkins.asterisk.org/jenkins/job/check-asterisk/1923/ |
| 16:58.44 | putnopvut | Thanks! |
| 17:01.47 | stefan27 | asterisk prioritizes video features much lower than audio right? It's primarily an audio platform? |
| 17:03.00 | file | it's not something anyone really focuses on, and during the last AstriDevCon it was mentioned that other projects are doing it well, and that it takes a lot to do that, so let them fill that piece |
| 17:05.08 | roramirez | guys, i doing when after restart, the reason pause will set for a member in app_queue |
| 17:05.30 | stefan27 | that makes sense, at the moment we are developing video services just because we can but we really lack significant amount of business cases |
| 17:06.13 | stefan27 | most normal people never communicate with video |
| 17:07.58 | file | if done well it can be neat |
| 17:08.22 | file | shows up each day at Digium in video form |
| 17:09.36 | putnopvut | file: that unit test is being a bit of a poophead, it appears. |
| 17:09.52 | file | putnopvut, pour diet coke on it |
| 17:10.03 | putnopvut | Good idea. |
| 17:10.26 | roramirez | in a function add_to_queue i think add a parameter when is called by reload_queue_members but i dont know if a elegant . What do you recomend me? |
| 17:14.06 | stefan27 | we have one customer which is basically services for a deaf community. (but asterisk is not doing the video-processing in that system). i wonder what the need really is for video under normal circumstances. why would you want to see the person you're talking to? |
| 17:14.26 | stefan27 | i guess video conferences makes sense when collaborating on something |
| 17:15.14 | file | collaboration and it also just establishes a deeper connection |
| 17:17.20 | stefan27 | I guess there are times I've called tech support queues where I would have wanted to actually show them stuff, but within a company you use teamviewer or similar tools for that anyways |
| 17:22.04 | stefan27 | and if it's a friend you need a video dialogue with, both probably use skype |
| 17:23.08 | stefan27 | but video development in itself is pretty fun, the behavior of app_confbridge with video was pretty hilarious, with one certain option set then whoever speaks the loudest is seen by everyone |
| 17:30.29 | roramirez | any suggestions? |
| 18:07.46 | *** join/#asterisk-dev newtonr (RustyNewto@nat/digium/x-sbvlatsjroscbunt) |
| 18:07.46 | *** mode/#asterisk-dev [+o newtonr] by ChanServ |
| 18:42.19 | putnopvut | file: I'm a confused-ass man regarding this failing unit test. The unit test creates three objects: blah-98joe, blah-93joe, and neener-93joe. I've confirmed through debug that we then search "id LIKE %blah-%" when attempting to retrieve items. For some reason, all three items get returned instead of just the first two. |
| 18:43.37 | file | hrm |
| 18:45.09 | file | what's the realtime_is_object_matching doing? |
| 18:45.27 | putnopvut | Haven't looked into that yet. |
| 18:46.18 | putnopvut | Ah, I think that's where the problem is. |
| 18:46.44 | putnopvut | The string is %blah-%, and the realtime_is_object_matching() function only looks at the content before the first % to see if there is a match. |
| 18:46.54 | putnopvut | This worked before because the string had previously been ^blah-% |
| 19:01.35 | *** join/#asterisk-dev roramirez (~rodrigo@181.75.126.119) |
| 19:19.06 | *** join/#asterisk-dev Demon_VoIP (~demon@ip253.net222.n37.ru) |
| 19:30.30 | Demon_VoIP | "make menuconfig" \ Compiler Flags \ DETECT_DEADLOCKS item is absent. What is the reason? :( |
| 19:38.53 | *** join/#asterisk-dev tzafrir (~tzafrir@bzq-179-40-172.cust.bezeqint.net) |
| 19:38.53 | *** mode/#asterisk-dev [+o tzafrir] by ChanServ |
| 19:42.58 | *** join/#asterisk-dev superscrat (~asanders@173-21-89-217.client.mchsi.com) |
| 19:43.48 | Demon_VoIP | i guess DETECT_DEADLOCKS in test menuselect only. |
| 20:14.42 | coreyfarrell | Demon_VoIP: it's included in menuselect by ./configure --enable-dev-mode |
| 20:18.57 | putnopvut | DETECT_DEADLOCKS tends to be pretty useless. The DEBUG_THREADS option, on the other hand, is much more useful since it creates the CLI "core show locks" command that can be useful for determining the cause of a deadlock. |
| 20:19.00 | *** join/#asterisk-dev gtjoseph (~gtj0@unaffiliated/gtj) |
| 20:21.12 | *** join/#asterisk-dev hexanol (~bibi@modemcable094.94-70-69.static.videotron.ca) |
| 20:29.42 | Demon_VoIP | putnopvut, DEBUG_THREADS && "core show locks" is already using. What do DETECT_DEADLOCKS in this case? Unfortunately, the server is hang, but I can identify only large UDP Rec-v |
| 20:30.33 | Demon_VoIP | Will DETECT_DEADLOCKS help me detect hang? |
| 20:31.35 | putnopvut | No, it probably will not. |
| 20:34.07 | Demon_VoIP | gtjoseph, today server hanged again with __ast_bt_get_addresses() from qualify_contact. Locks file exists.. |
| 20:34.38 | gtjoseph | <PROTECTED> |
| 20:35.12 | Demon_VoIP | 7 thread with the same location and (res_pjsip.c): MUTEX 3121 endpt_send_request req_wrapper |
| 20:35.28 | gtjoseph | that's actually OK... |
| 20:35.42 | gtjoseph | it means there were 7 qualifies in progress a the time of the crash |
| 20:36.05 | Demon_VoIP | Where was not any crash :) |
| 20:36.21 | gtjoseph | sorry...hang |
| 20:36.42 | gtjoseph | there was another lock i was concerned about though...in astdb |
| 20:37.21 | gtjoseph | and i'm trying to figure out of the hang and crash are related |
| 20:38.39 | gtjoseph | Demon_VoIP: add the new lock file to the ticket. Did you see my note about DETECT_DEADLOCKS? |
| 20:39.33 | Demon_VoIP | gtjoseph, just now i knew to have use --enable-dev-mode to enable DETECT_DEADLOCKS :) And: <putnopvut> DETECT_DEADLOCKS tends to be pretty useless |
| 20:40.03 | Demon_VoIP | <PROTECTED> |
| 20:40.36 | gtjoseph | i'll defer to putnopvut |
| 20:41.02 | gtjoseph | i'm just looking for any additional information. :) |
| 20:42.23 | putnopvut | DETECT_DEADLOCKS will sometimes print messages to the logs if it seems to be taking a while to grab a lock. It's not that helpful when it comes to actually trying to debug a deadlock. The DEBUG_THREADS option, however, makes Asterisk track which locks are being held by which threads, which makes it much easier to see the state of locks when a deadlock occurs. |
| 20:42.45 | Demon_VoIP | gtjoseph, debug-threads makes cpu works hard... Does DETECT_DEADLOCKS have any side-effect? |
| 20:42.56 | gtjoseph | yes it does. |
| 20:43.06 | gtjoseph | do if you're contrained, then don't turn it on |
| 20:43.21 | gtjoseph | s/do/so/ |
| 20:43.30 | coreyfarrell | putnopvut: DETECT_DEADLOCKS looks like it would be very spammy. if a deadlock ever actually happened it would log a message 5x per second? |
| 20:43.50 | gtjoseph | yeah, it's every 5th time it checks |
| 20:44.05 | gtjoseph | but i think the sleep is 200ms |
| 20:44.10 | putnopvut | 200 us |
| 20:44.12 | gtjoseph | so once per second |
| 20:44.20 | gtjoseph | oh, nevermind :) |
| 20:44.25 | gtjoseph | wasn't looking |
| 20:44.47 | putnopvut | Yeah, DETECT_DEADLOCKS is generally not helpful. |
| 20:45.10 | coreyfarrell | so wait 1000 messages per second?? |
| 20:45.16 | gtjoseph | :) |
| 20:45.33 | Demon_VoIP | logs are not problem.. "contact created", added and so on,,, tons of logs :) |
| 20:46.46 | Demon_VoIP | i'll try DETECT_DEADLOCKS on now |
| 20:47.03 | gtjoseph | be prepared to turn it off quickly :) |
| 20:47.31 | gtjoseph | at any one time, how many contacts do you have active? |
| 20:48.05 | Demon_VoIP | gtjoseph, how can i see this info? :) |
| 20:48.15 | gtjoseph | pjsip list contacts |
| 20:49.26 | Demon_VoIP | 445 |
| 20:49.50 | gtjoseph | so that explains why there are locks in the file for contact. that's not an issue |
| 20:50.14 | gtjoseph | 7 qualifies in progress for 445 contacts should be fine and they're not locked on each other |
| 20:51.08 | Demon_VoIP | ok. But server hangs, UDP queue not empty.. How can detect the reason? |
| 20:51.28 | Demon_VoIP | and UDP... and TCP |
| 20:51.37 | gtjoseph | the one lock file that specifically had an issue was against astdb |
| 20:52.37 | gtjoseph | do you use it specifically for something? |
| 20:52.50 | gtjoseph | if not, then check /var/lib/asterisk/astdb.sqlite |
| 20:53.03 | gtjoseph | if not, then check /var/lib/asterisk/astdb.sqlite3 |
| 20:54.56 | Demon_VoIP | check? astdb.sqlite3 and astdb.sqlite3-journal |
| 20:55.41 | gtjoseph | size? |
| 20:55.58 | gtjoseph | i'm not a sqlite3 expers |
| 20:56.04 | gtjoseph | s/expers/expert/ |
| 20:56.20 | Demon_VoIP | now size is: 621568 bytes |
| 20:57.01 | gtjoseph | anyone know what the -journal does? Temp file? |
| 20:57.16 | seanbright | probably a journal file |
| 20:57.22 | seanbright | (not to be a smart ass...) |
| 20:57.29 | seanbright | it's a database, so binary log? |
| 20:57.44 | Demon_VoIP | binary log for transaction |
| 20:58.47 | gtjoseph | that'd make sense |
| 20:58.59 | gtjoseph | did you attach the latest lock file? |
| 20:59.28 | Demon_VoIP | simetimes i see in astdb.sqlite3: |
| 20:59.29 | Demon_VoIP | PRAGMA foreign_keys=OFF; |
| 20:59.29 | Demon_VoIP | BEGIN TRANSACTION; |
| 20:59.30 | Demon_VoIP | ROLLBACK; -- due to errors |
| 21:00.37 | gtjoseph | well, if you don't mind losing the state of contact across restarts, you can stop asterisk, rename the file, then restart asterisk. |
| 21:05.24 | dan_j | Where can I learn more about testsuite? I've gone through the wiki but dont fully understand how to create a test. |
| 21:06.13 | gtjoseph | dan_j: the best way is to start with an existing test |
| 21:06.30 | dan_j | gtjoseph: when making a testsuite, do I checkout again so that it creates a separate gerrit commit? |
| 21:06.48 | dan_j | ah ignore that |
| 21:06.51 | dan_j | stupid question. |
| 21:07.06 | gtjoseph | yep. if you're in master, do a git checkout -b ASTERISK-xxxxx (same as the other review) |
| 21:07.34 | gtjoseph | do you have an idea of what the test needs to do (regardless of the mechanics) |
| 21:07.53 | gtjoseph | I.E. when you do it manually, what steps do you take? |
| 21:08.07 | Demon_VoIP | gtjoseph: added files. Two cases. Files: 2016_01_12__13_41_01.locks.txt 2016_01_12__13_42_01.locks.txt looks like the same. But 1 minute difference. |
| 21:08.26 | gtjoseph | is looking |
| 21:08.39 | coreyfarrell | dan_j: remember to run 'git review -s' on your clone of the testsuite repo. |
| 21:09.11 | dan_j | if I were testing it manually, start asterisk with regcontext=something, register an endpoint to asterisk, check for the existence of the extension in the dialplan 'dialplan show sipreg', then unregister and check that the extension no longer exists. |
| 21:09.25 | gtjoseph | perfect |
| 21:09.59 | gtjoseph | that's an easy test that yo can clone from another. give me a few minutes to look at Demon_VoIP's locks first. |
| 21:10.16 | dan_j | ok. thanks. |
| 21:10.36 | dan_j | coreyfarrell: do i need to do that whenever i change the machine i'm using? |
| 21:11.33 | dan_j | or only the first time I use that repo? |
| 21:11.43 | gtjoseph | dan_j: new repo |
| 21:12.21 | coreyfarrell | dan_j: you should run it each time you run 'git clone' on an asterisk.org repo. |
| 21:12.33 | dan_j | Ok. |
| 21:12.51 | gtjoseph | Demon_VoIP: the first 2 lock files look normal. The third shows a lock in dnsmgr which also looks normal. I'm just not seeing locking being the cause of the hangs. |
| 21:13.14 | gtjoseph | also, do you use chan_sip aling side pjsip? |
| 21:13.20 | gtjoseph | s/aling/along/ |
| 21:13.46 | gtjoseph | my typing skills deteriorate significantly after lunch i'm noticing |
| 21:13.59 | file | be aware that the locks you see will be a subset when PJSIP is in use, since PJSIP locks are not trackable by us |
| 21:14.11 | gtjoseph | yep, understood |
| 21:15.16 | gtjoseph | file: are yo aware of any issues with running sip and pjsip together as long as the ports are managed |
| 21:15.21 | file | no |
| 21:15.32 | file | https://issues.asterisk.org/jira/secure/attachment/53371/2016_01_10__22_20_01.backtrace-threads.txt shows a deadlock we fixed recently |
| 21:16.00 | Demon_VoIP | gtjoseph, chan_sip is used. A little peers witch have trouble with res_pjsip :( |
| 21:16.01 | gtjoseph | which thread |
| 21:16.07 | file | the monitor thread for PJSIP |
| 21:16.09 | gtjoseph | no worries |
| 21:16.24 | file | well - it may be that deadlock we fixed... |
| 21:16.40 | gtjoseph | git a thread # |
| 21:16.42 | file | the rest are all on a tsx timer callback it looks like |
| 21:16.46 | gtjoseph | got a thread # |
| 21:16.53 | file | thread 73 |
| 21:16.56 | gtjoseph | yeah those are pending options |
| 21:17.32 | file | you can be absolutely sure if it really is a deadlock by grabbing a backtrace of the running process, letting it go, and then grabbing another backtrace |
| 21:17.33 | file | and comparing |
| 21:18.26 | Demon_VoIP | deadlocks is the synonym of multithread :) |
| 21:19.08 | gtjoseph | file: when was the deadlock you're referring to fixed? |
| 21:19.18 | file | last week, Thursday maybe |
| 21:19.33 | gtjoseph | committed in 13? |
| 21:19.42 | file | https://gerrit.asterisk.org/#/c/1930/ |
| 21:20.22 | gtjoseph | Demon_VoIP: you pulled 13 yesterday, correct? |
| 21:21.19 | Demon_VoIP | gtjoseph, i do it regular. And hour ago ones again... Now i try to restart production |
| 21:21.40 | gtjoseph | ok, so it's probably not the issue in 1930 |
| 21:21.56 | file | generally it should be hard to stall the monitor thread for PJSIP, except for timers |
| 21:22.29 | Demon_VoIP | Whatchdog grab locks on hangs... and ones again after minutes. I can add backtrace to store. If it helps |
| 21:22.42 | gtjoseph | everything helps |
| 21:23.04 | Demon_VoIP | ok. done |
| 21:27.03 | Demon_VoIP | I'm nostalgic as I wrote and debugged a multi-threaded server corporate enterprise management system many years ago. Stress testing on many, many, many crazy-users. |
| 21:27.29 | seanbright | "multi-threaded server corporate enterprise management system" |
| 21:27.34 | seanbright | at least it was buzzword compliant |
| 21:34.20 | gtjoseph | Demon_VoIP: did you attach the files? Not seeing them |
| 21:35.22 | Demon_VoIP | gtjoseph, what files? Dmitriy Serov added a comment - 27 minutes ago More file of locks. Two hangs in 13:41 and 15:43 |
| 21:35.23 | Demon_VoIP | What's more? |
| 21:35.31 | Demon_VoIP | https://issues.asterisk.org/jira/browse/ASTERISK-25439 |
| 21:36.05 | gtjoseph | not sure, i thought you had another one from about 10 minutes ago |
| 21:36.12 | gtjoseph | if not, no worries. |
| 21:37.58 | Demon_VoIP | gtjoseph, no one. Server compiled on last branch 13. Tomorrow (for me) will be new logs of hangs with lock file and backtraces. And now i have to say thank you and bye. |
| 21:38.20 | gtjoseph | i'll keep looking. :) |
| 21:42.41 | gtjoseph | dan_j: so testsuite... |
| 21:43.09 | dan_j | Yep. Just looking at tests/channels/pjsip/registration/inbound/nominal/unregister/single_contact |
| 21:43.19 | dan_j | it registers and then unregisters |
| 21:43.26 | gtjoseph | i was just going to point you to that :) |
| 21:43.39 | gtjoseph | copy that directory |
| 21:43.54 | gtjoseph | unregister -> regcontext |
| 21:44.33 | dan_j | the whole unregister directory? with all the subfolders? |
| 21:44.54 | gtjoseph | oops, just single_contact |
| 21:45.06 | gtjoseph | up so it's a sibling of unregister |
| 21:45.57 | dan_j | where should i put it? tests/channels/pjsip/regcontext ? |
| 21:46.22 | gtjoseph | pjsip/registration/inbound/nominal |
| 21:46.38 | gtjoseph | sibling of unregister |
| 21:47.17 | gtjoseph | then edit tests/channels/pjsip/registration/inbound/nominal/tests.yaml and add the new directory |
| 21:49.08 | dan_j | done |
| 21:49.23 | gtjoseph | now before you do anything else, make sure it runs :) |
| 21:49.51 | gtjoseph | from the top level testsuite directory... |
| 21:50.05 | gtjoseph | ./rt -t tests/channels/pjsip/registration/inbound/nominal/regcontext |
| 21:50.28 | gtjoseph | ./runtests.py -t tests/channels/pjsip/registration/inbound/nominal/regcontext |
| 21:50.32 | gtjoseph | sorry |
| 21:51.32 | dan_j | I think i have to run the parent to that |
| 21:51.34 | dan_j | Failed to open tests/channels/pjsip/registration/inbound/nominal/regcontext/tests.yaml |
| 21:51.39 | dan_j | test.yaml is not there. |
| 21:51.46 | dan_j | tests.yaml |
| 21:51.54 | gtjoseph | did you edit the parent test.yaml |
| 21:51.58 | gtjoseph | in tests/channels/pjsip/registration/inbound/nominal |
| 21:51.58 | dan_j | I've got test-config.yaml |
| 21:52.04 | gtjoseph | that's fine |
| 21:52.40 | coreyfarrell | dan_j: in test.yaml you need to specify - test: 'regcontext' |
| 21:53.16 | coreyfarrell | did you use - dir: 'regcontext' ? |
| 21:53.26 | gtjoseph | oh yeah, my bad. |
| 21:54.03 | dan_j | coreyfarrell: yep |
| 21:54.32 | dan_j | oh. nope |
| 21:54.37 | gtjoseph | :) |
| 21:54.45 | dan_j | I added - dir: 'regcontext' to tests/channels/pjsip/registration/inbound/nominal/tests.yaml |
| 21:54.56 | dan_j | sorry. going nuts. |
| 21:55.09 | dan_j | thought you said 'did you use - test: 'regcontext' |
| 21:55.46 | dan_j | oh, lol. you did say that. |
| 21:57.23 | dan_j | Changed it to - test but got this |
| 21:57.24 | dan_j | http://pastebin.com/D2SbvMeQ |
| 21:57.43 | gtjoseph | yep, ok |
| 21:58.34 | gtjoseph | when you build asterisk you need to set the TEST_FRAMEWORK flag in cmenuselect compiler options |
| 21:58.53 | dan_j | ok. will do it now |
| 21:58.56 | gtjoseph | Compiler Flags |
| 21:59.03 | gtjoseph | all the way at the bottom |
| 21:59.41 | dan_j | yep |
| 22:02.49 | dan_j | Ok. it passed now |
| 22:03.02 | gtjoseph | it DID? |
| 22:03.05 | gtjoseph | wow |
| 22:03.58 | dan_j | also, ive added regcontext to the pjsip.conf file |
| 22:04.15 | gtjoseph | Ok, now edit your test-config.yaml and update the summary, description and minversion to 13.8.0 |
| 22:04.47 | coreyfarrell | rmudgett: would you mind doing a pastebin of example pj_dump_config output? |
| 22:04.58 | gtjoseph | i think you'll also have to add an extensions.conf |
| 22:05.39 | dan_j | true |
| 22:06.13 | gtjoseph | and we need to trim down the test-config.yaml |
| 22:08.26 | rmudgett | coreyfarrell: http://pastebin.com/UE9Ea8ks |
| 22:08.36 | dan_j | in extensions.conf, apart from the context name, do I need to make an extension too? For some reason, I added one to my server config. |
| 22:08.56 | rmudgett | First ever pastebin :) |
| 22:09.03 | gtjoseph | i don't *think* so |
| 22:09.10 | dan_j | ok. lets try without it |
| 22:10.01 | coreyfarrell | rmudgett: thanks. so I'm not that familiar with pjsip logging, it looks like unrelated pjsip logging messages could be caught in log_cb when show_info_active == 1? |
| 22:10.02 | dan_j | Ok. I'm back in test-config |
| 22:10.22 | gtjoseph | ok, look at the testobject-config section |
| 22:10.36 | dan_j | removed all the scenarios except for IPv4 & UDP |
| 22:10.43 | gtjoseph | perfect |
| 22:10.52 | gtjoseph | did you remove the extra 6 headermatches? |
| 22:11.03 | dan_j | Nope. didnt know what they were |
| 22:11.16 | gtjoseph | those are the events the test expects to get backi |
| 22:11.22 | rmudgett | coreyfarrell: Yes its possible. I needed to get the output enabled if it wasn't already enabled for the command output. |
| 22:11.34 | gtjoseph | there was a AOR_CONTACT_ADDED and AOR_CONTACT_REMOVED pair for each scenario |
| 22:11.48 | gtjoseph | we only need the first pair |
| 22:12.19 | dan_j | ok. done |
| 22:12.28 | gtjoseph | save and test |
| 22:13.33 | dan_j | yep. so just to review so far. the test is configured to add the extension when 'alice' registers. the test will register and unregister 'alice' while also checking that the AMI says that 'alice' is registered/unregistered. correct? |
| 22:13.58 | gtjoseph | exactly. when that's working we'll test for the extension add |
| 22:14.12 | dan_j | ok. running the test |
| 22:14.58 | coreyfarrell | rmudgett: hmm so that probably makes it unsafe to set 'show_info_active = a->fd' to use ast_cli from log_cb. |
| 22:15.34 | dan_j | gtjoseph: one sec. somethings gone wrong with my comments |
| 22:15.42 | gtjoseph | :) |
| 22:17.24 | rmudgett | That's why I have the disclaimer that the output goes to the DEBUG logger channel. |
| 22:18.01 | dan_j | gtjoseph: ok. passed now. |
| 22:18.15 | gtjoseph | it did? i was expecting a failure but ok. |
| 22:18.43 | dan_j | why a failure? it's just running a quarter of the original test |
| 22:18.54 | dan_j | 'tests/channels/pjsip/registration/inbound/nominal/regcontext', 'GIT-13-22801a0'] passed |
| 22:19.10 | gtjoseph | i was thinking it would fail because there was a ExtensionStatus message coming back |
| 22:19.19 | gtjoseph | no matter, let's add it |
| 22:19.40 | gtjoseph | so you now need to add another headermatch between the 2 existing ones |
| 22:20.20 | gtjoseph | Event: ExtensionStatus |
| 22:20.23 | coreyfarrell | rmudgett: seems like that means the following would not work? asterisk -rx 'pjsip show info' > pjsip-info.txt |
| 22:22.00 | rmudgett | coreyfarrell: Two lines would be output that say to look in the DEBUG output. |
| 22:22.27 | rmudgett | This isn't a command that would normally get used. |
| 22:22.52 | rmudgett | Only to check the config of pjproject |
| 22:22.53 | gtjoseph | dan_j: https://pastebin.com/5A103kfG |
| 22:23.13 | gtjoseph | rmudgett: actually i can use it as well |
| 22:23.27 | gtjoseph | if it can tell me how PJ_MAX_HOSTNAME is set |
| 22:23.39 | gtjoseph | at runtime |
| 22:23.39 | coreyfarrell | rmudgett: I was thinking that it would be useful for autosupport type of scripts |
| 22:23.41 | dan_j | gtjoseph: just looking at what the ami looks like when an extension is added. starting to understand |
| 22:23.57 | rmudgett | gtjoseph: It does have that listed. |
| 22:24.12 | gtjoseph | excellent, to you both |
| 22:25.47 | rmudgett | coreyfarrell: pjproject sends the output to its logger which is linked into Asterisk's logging system. |
| 22:26.03 | rmudgett | Not much choice in how to get the output. |
| 22:27.47 | rmudgett | It would have been perfect if pjproject returned a string. |
| 22:28.15 | rmudgett | like it does for pj_get_version() |
| 22:28.17 | coreyfarrell | rmudgett: I'm guessing there aren't functions to query these individual items? |
| 22:28.50 | gtjoseph | dan_j: 'manager show events' list all the events you can listen for. One of them is UserEvent which you can actually trigger from the dialplan. |
| 22:30.03 | rmudgett | coreyfarrell: No. Only the dump function in pjproject. |
| 22:30.24 | rmudgett | Better than nothing though. |
| 22:31.09 | coreyfarrell | agreed, just had to ask |
| 22:31.31 | gtjoseph | when i was looking at PJ_MAX_HOSTNAME yesterday, all i could find was to test the length of a structure in addr_resolv.h |
| 22:31.42 | gtjoseph | they don't make things easy |
| 22:34.01 | coreyfarrell | gtjoseph: technically you are testing the pjproject that asterisk was built against, not the one that is actually running. I mean in that case any difference would probably crash but in others maybe just act weird |
| 22:34.25 | *** join/#asterisk-dev [TK]D-Fender (~joe@64.235.216.2) |
| 22:34.30 | gtjoseph | yeah, if i just use a macro |
| 22:34.42 | gtjoseph | but i was thinking a little more creatively |
| 22:34.47 | coreyfarrell | or if you base it on the length of a structure |
| 22:37.06 | gtjoseph | coreyfarrell: i was going to take the address of ai_addr and subtract the address of ai_canonname. :) |
| 22:38.24 | dan_j | gtjoseph: recompiling. for some reason regcontext is not working. just checking i'm running code that includes it. |
| 22:38.35 | gtjoseph | cool |
| 22:39.36 | gtjoseph | i would have though that res_pjsip would have barfed if it saw an unrecognized option in globals |
| 22:40.26 | gtjoseph | you can use wireshark to spy on the ami messages |
| 22:48.55 | gtjoseph | dan_j: i'm also wondering if you might need a pause between the register an dunregister. |
| 22:50.23 | dan_j | still at it with this regcontext thing. |
| 22:50.48 | gtjoseph | no worries. it's still early for me. :) |
| 23:00.21 | dan_j | Ok. got my config sorted. wasnt a code issue. |
| 23:00.35 | gtjoseph | cool |
| 23:00.39 | dan_j | Connected to AMI, can see the TestEvent with AOR added. |
| 23:00.46 | dan_j | But dont see ExtensionState |
| 23:00.52 | gtjoseph | hmmm |
| 23:01.03 | dan_j | manager read=all |
| 23:01.07 | gtjoseph | yep |
| 23:01.21 | gtjoseph | how are you connected to ami? |
| 23:01.26 | dan_j | telnet |
| 23:01.36 | dan_j | so i can see everything |
| 23:01.36 | gtjoseph | ok |
| 23:02.06 | gtjoseph | can you see the event when you tets manually? |
| 23:03.19 | dan_j | I am testing manually. I see it added in the cli and when i do 'dialplan show sipregs', but not in the ami |
| 23:03.26 | dan_j | just double checking that again |
| 23:03.39 | gtjoseph | hmmm i see an event when i register |
| 23:03.47 | gtjoseph | for context default of course |
| 23:04.25 | dan_j | confirmed. doesnt appear in the AMI |
| 23:04.35 | gtjoseph | weird |
| 23:04.44 | gtjoseph | what happens if you don't use regcontext? |
| 23:04.54 | dan_j | what do you mean? |
| 23:05.01 | dan_j | then it wont create the extension |
| 23:05.23 | dan_j | wonder if i use regcontext=default |
| 23:06.37 | gtjoseph | nevermind. i use the pjsip_wizard which auto-creates hints |
| 23:06.43 | gtjoseph | that's why i see it |
| 23:13.45 | dan_j | oddly enough, when i do regcontext=default, the extension isnt added |
| 23:13.52 | dan_j | not sure why |
| 23:15.22 | gtjoseph | does regcontext attempt to create the context? |
| 23:16.32 | dan_j | nope. it needs to exist already |
| 23:17.05 | dan_j | the cli shows a warning when the endpoint unregisters, but not when it registers |
| 23:19.18 | dan_j | this is my code that adds the extension. |
| 23:19.30 | dan_j | https://www.irccloud.com/pastebin/hiP2fhFd/ |
| 23:19.58 | dan_j | Sorry. My irc client stopped me from pasting that in the channel. |
| 23:20.10 | dan_j | You got it? |
| 23:20.14 | gtjoseph | no prob. give me a minute, i want to check something |
| 23:22.07 | gtjoseph | i'm looking at how i did it in res_pjsip_config_wizard |
| 23:28.45 | dan_j | Can the test write to the AMI? |
| 23:29.09 | gtjoseph | yep |
| 23:29.20 | gtjoseph | it's a little different though |
| 23:30.22 | dan_j | So I've got two options is ExtensionState doesnt work. a) write to the AMI in order to check the extension exists. if thats possible. b) use the test to dial the extension in order to check it exists |
| 23:30.34 | dan_j | is that right? |
| 23:30.42 | gtjoseph | stupid question. are you adding a hint or a regular extension? |
| 23:31.21 | gtjoseph | and 'yes' |
| 23:32.48 | gtjoseph | look at pjsip/ami/show_* to see examples of writing to AMI |
| 23:34.19 | dan_j | regular. NoOp |
| 23:34.46 | gtjoseph | ok |
| 23:35.16 | dan_j | [ Context 'sipregs' created by 'pbx_config' ] |
| 23:35.16 | dan_j | <PROTECTED> |
| 23:35.33 | dan_j | the endpoint username is 7000 |
| 23:35.41 | gtjoseph | ok, justy not familiar with the scenario |
| 23:35.54 | rmudgett | gtjoseph: To change the PJ_MAX_HOSTNAME value you need to define it in the pjlib/pj/config_site.h file. Just adding it as a define on the command line won't help when Asterisk is compiled. |
| 23:36.24 | gtjoseph | the command line of Asterisk or pjproject? |
| 23:36.34 | rmudgett | pjproject |
| 23:36.54 | gtjoseph | no?? hmmm i thought the site config checked for pre-definition |
| 23:37.09 | dan_j | The idea is that DUNDI checks each server for the existence of that extension. Then it can figure out which server an endpoint is registered to. |
| 23:37.15 | gtjoseph | doesn't matter. it still defaults to 128 |
| 23:37.23 | rmudgett | the pj/config.h file checks for pre-definition |
| 23:37.25 | gtjoseph | dan_j: gotcha |
| 23:37.44 | coreyfarrell | rmudgett: I figured out a clean way to get pj_dump_config to use ast_cli |
| 23:38.53 | rmudgett | the pre-definition check in pj/config.h is after it includes pj/config_site.h |
| 23:38.53 | gtjoseph | dan_j: maybe the extensionstatus message is tied to devicestate whic is tied to hints |
| 23:39.03 | coreyfarrell | it all happens on a single thread.. handle_pjsip_show_info -> pj_dump_config -> pj_log -> log_cb |
| 23:39.59 | rmudgett | coreyfarrell: True but pj_log can be called by multiple threads |
| 23:40.01 | dan_j | gtjoseph: i see devicestate in the ami |
| 23:41.10 | gtjoseph | devicestate is triggered in the same function you put your code |
| 23:41.25 | gtjoseph | so that makes sense |
| 23:42.16 | gtjoseph | if you're sure your code is working as intended, i'd go for the 'make a call' |
| 23:42.32 | coreyfarrell | rmudgett: I'm aware of that.. |
| 23:43.33 | rmudgett | coreyfarrell: I see where you're going with the idea. |
| 23:43.50 | coreyfarrell | I'm going to post the patch I have on github so you can take a look |
| 23:44.52 | coreyfarrell | rmudgett: https://github.com/coreyfarrell/asterisk/commit/be41fc9d447c172713c904bb884bf3a3a742a2d3 |
| 23:45.19 | coreyfarrell | more complex, maybe better to use a struct to store all the statics in one but you get the idea |
| 23:45.42 | coreyfarrell | added the lock to prevent the signals from crossing paths, not sure that really matters but it seems right |
| 23:47.01 | coreyfarrell | in log_cb the check for show_info_fd != -1 is intended to optimize away the call to pthread_self() when possible |
| 23:50.13 | rmudgett | coreyfarrell: You need the lock to prevent multiple CLI instances from attempting the command at the same time. |
| 23:50.25 | rmudgett | I'll update my patch |
| 23:51.52 | coreyfarrell | rmudgett: up to you, might be appropriate to add that command to contrib/scripts/autosupport? |
| 23:52.16 | coreyfarrell | it already pulls a bunch of info from CLI if asterisk is running.. |
| 23:52.39 | rmudgett | ok, I'll do that too |
| 23:55.20 | dan_j | gtjoseph: isn't Ami Action ExtensionState supposed to tell me if an extension exists? |
| 23:55.32 | dan_j | I cant see the difference between when it exists and when it doesnt |
| 23:55.54 | gtjoseph | don't remember to be honest |
| 23:56.04 | coreyfarrell | dan_j: I think that might have to do with the hint attached to the extension? |
| 23:56.18 | coreyfarrell | sorry that was probably directed at gtjoseph and others |
| 23:56.43 | gtjoseph | i think that may be correct |
| 23:56.58 | dan_j | Ok. I'll go down the 'making a call' route then. |
| 23:57.12 | coreyfarrell | dan_j: there is another way |
| 23:57.27 | coreyfarrell | dan_j: use Getvar on the DIALPLAN_EXISTS function |
| 23:57.57 | coreyfarrell | make sure you add a dependency for the test on func_dialplan if you do that |
| 23:58.47 | dan_j | Ok. I'll give that a go tomorrow. Run out of time for tonight. |
| 23:58.49 | dan_j | Thanks |
| 23:59.54 | dan_j | gtjoseph: thanks for your help |