Weblogic OutOfMemory exception的误解 -- thread limitation
不是所有的OutofMemory exception都是内存问题。。。前几天有个客户的site报了以下错误:
[ERROR][thread ] Could not start thread Timer-72025. Resource temporarily unavailable Exception in thread "Timer-72024" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1097). Attempting to allocate 2G bytes There is insufficient native memory for the Java Runtime Environment to continue. Possible reasons: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hit Possible solutions: Reduce memory load on the system Increase physical memory or swap space Check if swap backing store is full Use 64 bit Java on a 64 bit OS Decrease Java heap size (-Xmx/-Xms) Decrease number of Java threads Decrease Java thread stack sizes (-Xss) Disable compressed references (-XXcompressedRefs=false) at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at java.util.Timer.<init>(Timer.java:137) at java.util.Timer.<init>(Timer.java:106) at ...... at java.util.TimerThread.mainLoop(Timer.java:512) at java.util.TimerThread.run(Timer.java:462)
第一反应以为是内存不足,于是dump了heap回来看,发现系统只占用了80M的内存,远远系统内存绰绰有余,那么为啥会出现这种情况呢?查看了出错的代码,发现代码启动了很多的线程,遂想起以前遇到过类似的问题,weblogic里面启动的线程太多,超出了linux的最大限制,也是会出这个看起来内存泄漏的日志。查看系统的限制:
sc-1@vECE-222 limits.d # ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 38610 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 8192 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited发现最大的进程数是1024,注意:linux里面计算线程和进程是用一样计算的,这里说的max user processes是包括线程的。
使用以下代码重现问题:
public class ThreadExample { public static void main(String[] args){ System.out.println(Thread.currentThread().getName()); for(int i=0; i< 2048; i++){ new Thread("" + i){ public void run(){ System.out.println(Thread.currentThread().getName() + " running"); try { Thread.sleep(60000); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println(Thread.currentThread().getName() + " ending"); } }.start(); } } }执行和输出:
sc-1@ECE30CP3_Z1 etmdaha # su -p occas -c "/opt/jrockit-jdk/bin/java ThreadExample" Main Thread 0 running 5 running // 836 running [ERROR][thread ] Could not start thread 837. Resource temporarily unavailable Exception in thread "Main Thread" java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096). Attempting to allocate 3G bytes There is insufficient native memory for the Java Runtime Environment to continue. Possible reasons: The system is out of physical RAM or swap space In 32 bit mode, the process size limit was hit Possible solutions: <strong>Reduce memory load on the system Increase physical memory or swap space Check if swap backing store is full Use 64 bit Java on a 64 bit OS Decrease Java heap size (-Xmx/-Xms) Decrease number of Java threads Decrease Java thread stack sizes (-Xss) Disable compressed references (-XXcompressedRefs=false) </strong> at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:640) at ThreadExample.main(ThreadExample.java:6)
linux系统资源有各种限制:内存限制,栈限制,文件数目限制,线程限制,要小心陷进这些问题。
# ulimit -a core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 79010 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 1024 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
这里有两篇很好的文章:
http://people.redhat.com/alikins/system_tuning.html#threads
http://kavassalis.com/2011/03/linux-and-the-maximum-number-of-processes-threads/
Weblogic OutOfMemory exception的误解 -- thread limitation,古老的榕树,5-wow.com
郑重声明:本站内容如果来自互联网及其他传播媒体,其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享,并不代表本站赞同其观点和对其真实性负责,也不构成任何其他建议。