| 00:25.32 | *** join/#tomcat acidjnk (i=acid@p5B3F0642.dip0.t-ipconnect.de) |
| 00:59.22 | *** join/#tomcat tomisina (n=tomisina@CMU-218271.WV.CC.cmu.edu) |
| 01:03.38 | *** join/#tomcat randrewj (n=raj-user@user-160u617.cable.mindspring.com) |
| 01:13.48 | *** join/#tomcat acidjnk (i=acid@p5B3F04A4.dip0.t-ipconnect.de) |
| 01:28.25 | *** join/#tomcat tomisina (n=tomisina@CMU-218271.WV.CC.cmu.edu) |
| 01:52.39 | tomisina | hey what |
| 01:52.53 | tomisina | 's so bad about storing tons of information in the session |
| 01:53.05 | tomisina | like information that's being pushed to the view... |
| 01:54.21 | jasonb | Nothing except it makes the request heavier whenever it gets persisted to disk, or read back from disk. But, if you can't limit how much memory that takes, you'll likely find yourself in an out of memory situation in the JVM. |
| 01:54.51 | jasonb | It's generally better to keep it stored in the DB, but not always. |
| 02:11.43 | pucko | it all depends on how heavy traffic one have. lots of users with lots of data in sessions = out of memory sooner or later |
| 02:14.07 | *** join/#tomcat acidjnk (i=acid@p5B3F0184.dip0.t-ipconnect.de) |
| 02:15.30 | pucko | if one absolutely want to cache stuff in memory, only use sessions when the data differs between users and put the rest in a applicationwide object |
| 02:39.27 | *** join/#tomcat acidjnk (i=acid@p5B3F430E.dip0.t-ipconnect.de) |
| 03:06.12 | *** join/#tomcat tomisina (n=tomisina@CMU-218271.WV.CC.cmu.edu) |
| 03:28.10 | *** join/#tomcat acidjnk (i=acid@p5B3F4044.dip0.t-ipconnect.de) |
| 03:37.34 | *** part/#tomcat randrewj (n=raj-user@user-160u617.cable.mindspring.com) |
| 03:49.10 | *** join/#tomcat acidjnk (i=acid@p5B3F3E65.dip0.t-ipconnect.de) |
| 03:52.33 | *** join/#tomcat isa (n=ighigiu@cpe-68-206-97-180.satx.res.rr.com) |
| 04:21.09 | *** join/#tomcat acidjnk (i=acid@p5B3F3CD2.dip0.t-ipconnect.de) |
| 04:34.44 | *** join/#tomcat acidjnk (i=acid@p5B3F3A18.dip0.t-ipconnect.de) |
| 04:57.07 | *** join/#tomcat CharlieS1 (n=charlie@pool-71-114-232-73.austtx.dsl-w.verizon.net) |
| 05:21.40 | *** join/#tomcat odin_ (n=dlm@host81-132-27-142.range81-132.btcentralplus.com) |
| 05:38.38 | *** join/#tomcat acidjnk (i=acid@p5B3F3863.dip0.t-ipconnect.de) |
| 06:48.32 | *** join/#tomcat cofeineSunshine1 (n=justinas@212.47.107.24) |
| 07:27.44 | *** join/#tomcat cofeineSunshine (n=justinas@212.47.107.24) |
| 07:39.02 | *** join/#tomcat kjkoster5489 (n=kjkoster@kjkoster.org) |
| 08:39.31 | *** join/#tomcat _eth0 (i=schlamak@f049010208.adsl.alicedsl.de) |
| 08:59.15 | *** join/#tomcat Kassec (n=Kassec@gar78-1-82-238-169-95.fbx.proxad.net) |
| 09:02.13 | Kassec | hi ! |
| 09:02.19 | *** join/#tomcat TBBle (n=tbble@202.55.155.45) |
| 09:11.50 | Kassec | I have an issue with a tomcat 5.5.23, jvm 1.5.0_12-b04, Debian Linux 2.6.18, spring 1.2 : some requests take hours while most are served in just a few ms. Tomcat's manager shows some requests in servicing status that are more than 4,000,000 ms old. We tried to remove any front-end (like apache/mod_jk) but it does not help. Requests are still in servicing status while the tcp socket has already been closed for a while in the Linux's tcp |
| 09:11.57 | Kassec | any hint welcome ;) |
| 09:26.37 | *** join/#tomcat a4akba (n=a4akb@62.215.156.215) |
| 09:33.09 | kjkoster5489 | Kassec: Have you traced some of the requests in Java already? |
| 09:37.27 | *** join/#tomcat teszrr (n=ISADMIN@62.215.156.215) |
| 09:39.33 | *** join/#tomcat a4akba (n=a4akb@62.215.156.215) |
| 09:39.39 | a4akba | hola |
| 09:50.43 | *** join/#tomcat KCNV (n=polx@62.128.48.166.static.012.net.il) |
| 09:58.05 | Kassec | kjkoster5489: nope, it's not my project, I'm a sysadmin (with java knowledge), not the dev of this project |
| 10:00.36 | Kassec | and the volume is large enough (50 reqs/sec) so that debug traces will be a real pain ;) |
| 10:07.29 | KCNV | hey guys. I'm running Tomcat 5.5.23, JRE 1.4.2_03 on a Win2003 RC2 server. I have a problem that when I start-up the server and try to access the http://localhost:8080 page, I get a 404 error. I checked the logs and there seems to be nothing there but notifications of the start-up and shut-down of the server. I have also configured my manager and admin users, and made sure the connector for is set to the 8080 port. What else can I try to fix it |
| 10:43.23 | *** join/#tomcat kjkoster5489 (n=kjkoster@kjkoster.org) |
| 11:10.55 | kjkoster5489 | Kassec: is it limited to a specific request url or not? You could enable request response time logging in the logs to see. |
| 11:14.35 | *** join/#tomcat wild_oscar (n=miguel@a83-132-60-128.cpe.netcabo.pt) |
| 11:18.08 | Kassec | kjkoster5489: not a specific url, we have two or three candidates over 150 that are mostly causing but they are, by far, the most requested so I believe this is fair ... |
| 11:19.04 | kjkoster5489 | Hmm. |
| 11:19.42 | kjkoster5489 | The question is what is happening here, assuming the application is at fault and not some proxy or httpd config problem. |
| 11:19.57 | kjkoster5489 | Are these straight request for some content? |
| 11:20.09 | kjkoster5489 | Your basic JSP with a database to get data from? |
| 11:20.16 | kjkoster5489 | Or web services as backend? |
| 11:20.50 | *** join/#tomcat Nicke (n=niclasa@ua-83-227-140-135.cust.bredbandsbolaget.se) |
| 11:21.09 | kjkoster5489 | An idea might be to do occasional thread dumps to see what your threads are blocking on. |
| 11:24.55 | Kassec | kjkoster5489: I really have no idea. I already spend some hours trying to see something. We really optimized the db server, changed a little the tomcat config, etc. |
| 11:25.03 | Kassec | we had some success on perf when everything is fine |
| 11:25.19 | kjkoster5489 | I don't understand that last statement. |
| 11:25.44 | Kassec | of course ;) |
| 11:25.48 | kjkoster5489 | You have no idea of the application architecture? |
| 11:25.59 | Kassec | I mean, we now have better app perf for pages served correctly |
| 11:25.59 | kjkoster5489 | Don't know what web services it uses? |
| 11:26.32 | Kassec | and then, after some time (10 to 30 minutes after tomcat's restart), we begin to "loose" some threads and they get stuck |
| 11:26.47 | Kassec | and the client gets no response (or after one hour !) |
| 11:26.59 | kjkoster5489 | Ok. |
| 11:27.04 | kjkoster5489 | No errors in the logs? |
| 11:27.18 | kjkoster5489 | You have apache in front of Tomcat? |
| 11:27.27 | Kassec | a few app exceptions, nothing meaningfull |
| 11:27.34 | Kassec | no more apache since a few hours |
| 11:27.57 | Kassec | up to know, I wasn't betting on an app problem |
| 11:28.22 | Kassec | I don't understand that tomcat doesn't timeout its threads after some time |
| 11:28.30 | Kassec | and didn't find a way to configure that |
| 11:28.45 | Kassec | that would, at least, let us run the service, even with bugs |
| 11:28.59 | kjkoster5489 | Yes, I see your point. |
| 11:29.01 | Kassec | but now, with the number of clients, we exhaust the 1500 threads pool quite quickly |
| 11:29.10 | kjkoster5489 | Hmm. |
| 11:29.20 | kjkoster5489 | Have you considered load balancing? |
| 11:29.41 | Kassec | the first setup was running laod balancing over two tomcats using mod_jk |
| 11:29.51 | kjkoster5489 | Not to solve the issue, but to lighten the load on your servers, allowing it to run longer until the thread statt to die.\ |
| 11:30.14 | kjkoster5489 | And you have the same behaviour with and without the load balancing? |
| 11:30.21 | Kassec | I'd better have 3000 threads in the tomcat, would do the same, I only have one server for serving the app right now |
| 11:30.25 | Kassec | exactly |
| 11:30.42 | Kassec | on both tomcats |
| 11:30.54 | kjkoster5489 | No memory issues? |
| 11:31.09 | Kassec | Free memory: 78.22 MB Total memory: 531.93 MB Max memory: 986.12 MB |
| 11:31.34 | Kassec | no file desc issue either, ulimit has been raised |
| 11:31.50 | kjkoster5489 | Have you enabled JMX, so that you can use jconsole to look into the server? |
| 11:32.28 | kjkoster5489 | I have relatively littel experience doing so, but I think you should cause thread dumps on the server using kill -3 iirc. |
| 11:32.54 | Kassec | I'm going to at the moment, would you help find the values where I could have the better info ? |
| 11:32.58 | kjkoster5489 | I'm not sure doing so is safe from a functional persecive, but it may help you locate what the dead threads are and what they are doing. |
| 11:33.11 | Kassec | thread dump is a great idea, I didn't thought about it ! |
| 11:33.40 | Kassec | any chance I can get this info through jmx ? |
| 11:33.43 | kjkoster5489 | Just start tomcat with -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false |
| 11:33.57 | kjkoster5489 | You are running with -server enabled, yes? |
| 11:34.07 | Kassec | hum, let me check |
| 11:34.29 | kjkoster5489 | I'm not sure that the client hotspot uses more than one cpu. |
| 11:35.13 | Kassec | nope |
| 11:36.02 | Kassec | will relaunch the app with that -server and jmx enabled soon |
| 11:37.50 | kjkoster5489 | What monitoring tools do you use? Just curious. |
| 11:38.07 | Kassec | for ? |
| 11:38.21 | Kassec | systems ? |
| 11:40.40 | *** join/#tomcat LongBeach (n=mike@AFontenayssB-152-1-57-48.w82-121.abo.wanadoo.fr) |
| 11:48.52 | kjkoster5489 | Kassec: yes |
| 11:53.16 | Kassec | kjkoster5489: jmx gives access to threads stack trace, it's great. |
| 11:53.35 | kjkoster5489 | :-) |
| 11:53.52 | kjkoster5489 | <golllum>JMX is my friend.</gollum> |
| 11:55.14 | kjkoster5489 | You may have to script things, though. |
| 11:55.30 | kjkoster5489 | If the volume is as high as you say. |
| 12:00.29 | Kassec | I just found that the dev lied to me |
| 12:00.47 | Kassec | when I asked 'no more mtex in the code ?' and they said no |
| 12:01.03 | Kassec | And now we found a lot of synchronized methods |
| 12:01.05 | Kassec | ... |
| 12:06.31 | kjkoster5489 | Well, it's kind'a hard to write an app without them. |
| 12:06.57 | kjkoster5489 | Are you sure they are not library methods? |
| 12:38.25 | Kassec | kjkoster5489: they are app methods |
| 12:38.32 | kjkoster5489 | Ok. |
| 12:38.45 | Kassec | They're checking the code to see how to avoid them, or at least deadlocks |
| 12:38.53 | Kassec | thanks a lot for your time |
| 12:39.18 | kjkoster5489 | In my experience you can save a lot of this by making use of the synchornised library classes. |
| 12:39.28 | kjkoster5489 | BlockingQueue is my favourite. |
| 12:40.12 | Kassec | yes, I never use synchronized myself, too much trouble and no timeout ;-) |
| 12:40.20 | kjkoster5489 | :-) |
| 12:40.39 | kjkoster5489 | Then you'll like the blocking queue. It has time-outs. |
| 12:40.47 | Kassec | but here, I was given a problem on a project that I didn't know it existed a few hours ago ! |
| 12:41.02 | kjkoster5489 | Such is life. |
| 12:41.12 | kjkoster5489 | What systems monitoring tool do you use? |
| 12:41.29 | Kassec | oh yes, several |
| 12:41.31 | Kassec | munin |
| 12:41.35 | kjkoster5489 | Zabbix? |
| 12:41.38 | Kassec | nagios+centreon |
| 12:41.40 | Kassec | and our owns |
| 12:41.47 | kjkoster5489 | ok. |
| 12:41.48 | Kassec | developped in j2ee |
| 12:42.14 | kjkoster5489 | What JMX attributes do you monitor on a regular basis? |
| 12:42.17 | Kassec | I don't really know zabbix |
| 12:42.32 | kjkoster5489 | I'm working on a Java plugin for Zabbix. |
| 12:42.46 | kjkoster5489 | It's just another monitor tool. :) |
| 12:42.48 | Kassec | you're working for Zabbix project ? |
| 12:42.53 | kjkoster5489 | No, my own. |
| 12:42.53 | Kassec | is it cool ? |
| 12:43.06 | kjkoster5489 | Well, Java monitoring with Zabbix is vool. |
| 12:43.11 | kjkoster5489 | s/vool/cool/ |
| 12:43.17 | Kassec | will give it a try |
| 12:43.28 | kjkoster5489 | http://www.kjkoster.org/zapcat/ |
| 12:43.34 | kjkoster5489 | That's the Java plugin. |
| 12:43.47 | kjkoster5489 | I'm working on a template. |
| 12:44.01 | kjkoster5489 | So I would like to know what you monitor from Tomcat and JVM's. |
| 12:44.33 | kjkoster5489 | If you try Zabbix, drop me a line and I will e-mail you the draft Java template. |
| 12:46.32 | Kassec | ok. we mostly monitor tomcat's nb threads, connections, reqs, etc. |
| 12:46.35 | Kassec | very usual |
| 12:46.46 | Kassec | digging to find interesting is long |
| 12:46.57 | Kassec | it takes too much time for usual needs |
| 12:47.35 | Kassec | having a tool preconfigured with a bunch of good templates that give access to good beans to monitor would be very cool |
| 12:47.42 | Kassec | would save time and improve monitoring |
| 12:49.28 | kjkoster5489 | Ok. threads have in my template, memory (os, Java heap and Java nonheap) and file descritors. |
| 12:49.38 | kjkoster5489 | How do you monitor the connections? |
| 12:49.44 | kjkoster5489 | And the requests? |
| 12:59.23 | *** join/#tomcat ries (n=ries@200.110.78.134) |
| 12:59.25 | Kassec | don't remember the values |
| 12:59.30 | Kassec | they're exposed by tomcat |
| 13:03.24 | kjkoster5489 | ok. |
| 13:07.23 | Kassec | kjkoster5489: |
| 13:07.24 | Kassec | ajp_requests.jmxObjectName Catalina:name=jk-8009,type=GlobalRequestProcessor |
| 13:07.24 | Kassec | ajp_requests.jmxAttributeName requestCount |
| 13:07.36 | Kassec | ajp_errors.jmxAttributeName errorCount |
| 13:07.52 | kjkoster5489 | Thanks. |
| 13:08.09 | Kassec | catalina_threads_busy.jmxObjectName Catalina:name=ajp-8009,type=ThreadPool |
| 13:08.09 | Kassec | catalina_threads_busy.jmxAttributeName currentThreadsBusy |
| 13:08.15 | Kassec | catalina_threads_count.jmxObjectName Catalina:name=ajp-8009,type=ThreadPool |
| 13:08.16 | Kassec | catalina_threads_count.jmxAttributeName currentThreadCount |
| 13:08.35 | Kassec | catalina_bytes_received.jmxObjectName Catalina:name=ajp-8009,type=GlobalRequestProcessor |
| 13:08.35 | Kassec | catalina_bytes_received.jmxAttributeName bytesReceived |
| 13:08.41 | Kassec | catalina_bytes_sent.jmxObjectName Catalina:name=ajp-8009,type=GlobalRequestProcessor |
| 13:08.41 | Kassec | catalina_bytes_sent.jmxAttributeName bytesSent |
| 13:08.43 | Kassec | etc. |
| 13:19.20 | kjkoster5489 | What are those requests to /p-80.html in my access logs? |
| 13:32.02 | BULLE | kjkoster5489: no idea, what does your p-80.html contain ? |
| 13:32.25 | kjkoster5489 | Well, it does not exist, but someone is requesting it. |
| 13:32.47 | kjkoster5489 | I found them in my 404 log. |
| 13:37.41 | BULLE | guess you have to ask the one who requests it what he/she expects to get hold of |
| 13:38.04 | BULLE | might also be some known security issue in some framework, someone is trying to exploit, and just probes all sites they can find |
| 13:40.09 | kjkoster5489 | I was wondering if it may be the latter. |
| 13:40.33 | kjkoster5489 | I can't find anything in that direction through Google. |
| 13:40.54 | BULLE | neither can i |
| 13:41.09 | BULLE | i dont stay up to date when it comes to security issues though |
| 13:41.16 | BULLE | but one would think google would come up wth something |
| 13:46.46 | *** join/#tomcat acidjnk (i=acid@p5B3F3863.dip0.t-ipconnect.de) |
| 14:11.06 | *** join/#tomcat frankbille (n=frankbil@apache/committer/frankbille) |
| 14:14.00 | frankbille | (Tomcat: 5.5.25, Java:; |
| 14:14.14 | frankbille | 1.5.0_13 |
| 14:15.04 | frankbille | wups, damn keyboard... OS: Linux) Question: Is it possible to specify in which order the virtual hosts start up? |
| 14:16.42 | frankbille | I have tried: reordering in server.xml, renaming the host folder and webapp folder so they are named in natural sorting order of what I want |
| 14:17.56 | frankbille | and I have of cause tried to search the lists/web |
| 14:31.21 | frankbille | Hmm, according to this thread it's not really possible: http://markmail.org/message/pb5thhnfbwrnhzw3 |
| 14:31.26 | frankbille | Thank you for your time |
| 15:50.06 | *** join/#tomcat vinse_ (n=vinse_@208.253.223.146) |
| 16:03.18 | *** join/#tomcat jasonb (i=noneoyer@adsl-66-124-73-250.dsl.sntc01.pacbell.net) |
| 16:06.44 | wild_oscar | jasonb, glad I find u |
| 16:06.58 | wild_oscar | perhaps u can easily explain a doubt I had yesterday |
| 16:07.44 | wild_oscar | regarding the reloadable="true" option |
| 16:08.03 | wild_oscar | tomcat has to reload the entire web app when it detects a class change, correct? |
| 16:12.52 | jasonb | It reloads all class files. |
| 16:13.16 | jasonb | So, all servlets in the webapp must be taken out of service, de-referenced, reloaded, and reinstantiated. |
| 18:00.34 | *** join/#tomcat ibot (i=ibot@pdpc/supporter/active/TimRiker/bot/apt) |
| 18:00.34 | *** topic/#tomcat is Stable versions: 6.0.14, 5.5.23 and 4.1.36. Newbies use the official binary from tomcat.apache.org, or an RPM package from http://www.webdroid.org:8080/archives/tomcat-package. Check your Tomcat logs before you ask for an answer. SLOW MOTION CHANNEL (we all have jobs & kids): Ask your question, including your TC, Java, & OS versions, then wait; check back often for answers. |
| 18:05.14 | *** join/#tomcat Kassec (n=Kassec@gar78-1-82-238-169-95.fbx.proxad.net) |
| 19:32.30 | *** join/#tomcat chickenFuego (n=chicken@Le723.l.pppool.de) |
| 20:45.25 | *** join/#tomcat wltjr (n=wltjr@gentoo/developer/wltjr) |
| 21:00.14 | *** join/#tomcat fzlogik (n=fzlogik@host-84-9-147-86.bulldogdsl.com) |
| 21:39.31 | *** join/#tomcat Kukoc (i=Kukoc@host32-116-dynamic.6-79-r.retail.telecomitalia.it) |
| 21:39.35 | *** part/#tomcat Kukoc (i=Kukoc@host32-116-dynamic.6-79-r.retail.telecomitalia.it) |
| 21:40.14 | *** join/#tomcat vanksi (n=vanksi@stekt2.oulu.fi) |
| 21:55.27 | *** part/#tomcat wild_oscar (n=miguel@a83-132-60-128.cpe.netcabo.pt) |
| 22:41.52 | *** join/#tomcat ries (n=ries@200.110.78.134) |