java not enough memory error.
|
After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
12 posts by 7 authors
|
|
|
Patrick Heppler |
10/28/13
|
Hi,
I'm running railo 4.1.1.009 on a centos box. Today I updated JRE (Oracle) from jre-7_21 to jre-7_45. And now when starting jetty I get:
Starting Jetty: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000007e0000000, 536870912, 2097152, 0) failed; error='Cannot allocate memory' (errno=12)
With jre-7_21 I had no issue. Railo is running fine, anyway.
My Java Args (from a Railo Blog ;) ):
-server -Djava.io.tmpdir=/dev/shm -javaagent:/opt/jetty/lib/ext/railo-inst.jar -Dsun.io.useCanonCaches=false -Xms1024m -Xmx2048m -XX:PermSize=512M -XX:MaxPermSize=512M -XX:NewSize=512M -XX:MaxNewSize=512M -XX:GCTimeRatio=5 -XX:ThreadPriorityPolicy=42 -XX:ParallelGCThreads=4 -XX:+UseParNewGC -XX:MaxGCPauseMillis=50 -XX:+DisableExplicitGC -XX:MaxHeapFreeRatio=70 -XX:MinHeapFreeRatio=40 -XX:+UseStringCache -XX:+OptimizeStringConcat -XX:+UseTLAB -XX:+ScavengeBeforeFullGC -XX:CompileThreshold=1500 -XX:+TieredCompilation -XX:+UseBiasedLocking -Xverify:none -XX:+UseThreadPriorities -XX:+UseLargePages -XX:LargePageSizeInBytes=2m -XX:+UseFastAccessorMethods -XX:+UseCompressedOops -Djava.security.egd=/dev/urandom -Dfile.encoding=UTF-8 -Duser.language=de -Duser.region=DE -Duser.timezone=Europe/Berlin -Dcoldfusion.jsafe.defaultalgo=FIPS186Random
|
|
Phil Sweeney |
11/2/13
|
To be honest, I think using a bunch of JVM flags from blogs/google searches is a bad idea, and is potentially why you have run into this problem.
When I went through this exercise late last year, I researched each flag I saw mentioned around the place. Most of them were already set as default in modern JVMs and others were simply misunderstood.
For our high traffic (and now very fast) site, this is all I have for my JVMs:
export CATALINA_OPTS="-Xms768m \
-Xmx768m \
-XX:NewSize=448m \
-XX:MaxNewSize=448m \
-XX:MaxTenuringThreshold=32 \
-XX:MaxPermSize=128m \
-XX:+UseConcMarkSweepGC \
-XX:+CMSClassUnloadingEnabled \
-XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses \
The New size relates to the nature of the site, in that it has much more temporary allocations than permanent. But I did that based on analysis with visualvm and then testing.
Obviously you would choose your memory sizes based on your site profile.
But every single one is there for a demonstrable reason.
Phil
When I went through this exercise late last year, I researched each flag I saw mentioned around the place. Most of them were already set as default in modern JVMs and others were simply misunderstood.
For our high traffic (and now very fast) site, this is all I have for my JVMs:
export CATALINA_OPTS="-Xms768m \
-Xmx768m \
-XX:NewSize=448m \
-XX:MaxNewSize=448m \
-XX:MaxTenuringThreshold=32 \
-XX:MaxPermSize=128m \
-XX:+UseConcMarkSweepGC \
-XX:+CMSClassUnloadingEnabled \
-XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses \
The New size relates to the nature of the site, in that it has much more temporary allocations than permanent. But I did that based on analysis with visualvm and then testing.
Obviously you would choose your memory sizes based on your site profile.
But every single one is there for a demonstrable reason.
Phil
- show quoted text -
|
|
Igal |
11/2/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
I think using a bunch of JVM flags from blogs/google searches is a bad idea, and is potentially why you have run into this problem.
but having said that I think that you also set some flags that you shouldn't mess with (like MaxNewSize and MaxTenuringThreshold) unless you have a very good reason to do so.
I recommend setting the following flags:
-Xms={minimum-heap-memory-in-mb} m
-Xmx={maximum-heap-memory-in-mb}m
-XX:MaxPermSize ={maximum-perm-gen-in-mb} m
-XX:+HeapDumpOnOutOfMemoryError
the memory values should be set according to physical RAM size. MaxPermSize should be 256m if possible (though 128m can be used as minimum), and -Xmx should be between 1024m and 4096m, again, depends on available RAM.
- show quoted text -
--- show quoted text -
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/88afc251-7daa-4ad4-8d1b-bcd7a651ea3b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
-- Igal Sapir Railo Core Developer http://getRailo.org/
|
|
Patrick Heppler |
11/4/13
|
Just FYI
https://www.farbeyondcode.com/Optimizing-Tomcat-s-JVM-Options-for-Railo-4-1-5-3001.html
This is where I got this settings from. But I'll try the JVM Settings recommended
Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick Heppler:
Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick Heppler:
- show quoted text -
|
|
Kai Koenig |
11/5/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
That's fine, but be aware and warned that JVM/GC settings are always heavily depending on one's hardware, OS, configuration, application as well as _optimisation goals_.
I strongly advise against copying any sort of JVM/GC settings from anyone's blog or tutorial. They might really well work for Bruce's setup but they might (and apparently do) cause issue for you.
Cheers
Kai
- show quoted text -
|
|
Igal |
11/5/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
This is where I got this settings from
- show quoted text -
- show quoted text ---To view this discussion on the web visit https://groups.google.com/d/msgid/railo/585C7698-F6A1-4422-9DA1-D26019A8AE9D%40gmail.com.
Did you find this reply useful? Help the Railo community and add it to the Railo Server wiki at https://github.com/getrailo/railo/wiki
---
You received this message because you are subscribed to the Google Groups "Railo" group.
For more options, visit https://groups.google.com/groups/opt_out.
-- Igal Sapir Railo Core Developer http://getRailo.org/
|
|
Patrick Heppler |
11/11/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
I now switched back from custom jetty setup to railo/tomcat (installer bundle) because I had an issue with javaagent.
Am Montag, 4. November 2013 20:40:06 UTC+1 schrieb Igal:
I now have this Java Opts:
JAVA_OPTS="-server -Djava.io.tmpdir=/dev/shm -javaagent:/usr/share/nginx/html/
example.com/WEB-INF/lib/railo-inst.jar -Dsun.io.useCanonCaches=false -Xms1024m -Xmx2048m -XX:MaxPermSize=256M -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=/dev/urandom -Dfile.encoding=UTF-8 -Duser.language=de -Duser.region=DE -Duser.timezone=Europe/Berlin -Dcoldfusion.jsafe.defaultalgo=FIPS186Random";
I still get the memory errors, not on railo start but on restart. Linux top command shows more then 1.4 GB free while railo is running.
Railo Server Admin shows 30% Heap and 20% Non-Heap used.
Am Montag, 4. November 2013 20:40:06 UTC+1 schrieb Igal:
- show quoted text -
|
|
Chris Blackwell |
11/11/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
Patrick (and Bruce)
Do not use JAVA_OPTS to pass arguments to tomcat, use CATALINA_OPTS
for an explanation of why, see ->
http://www.wellho.net/mouth/2163_CATALINA-OPTS-v-JAVA-OPTS-What-is-the-difference-.html
basically, if you set -Xms1024m in JAVA_OPTS the java process that is spawned to restart tomcat also uses that amount of memory, so only use global options in JAVA_OPTS, not per process stuff which for tomcat should be placed in CATALINA_OPTS.
Chris
- show quoted text -
To view this discussion on the web visit https://groups.google.com/d/msgid/railo/39c735b9-e2ee-4486-8647-ee7f25621066%40googlegroups.com.- show quoted text -
For more options, visit https://groups.google.com/groups/opt_out.
|
|
Jordan Michaels |
11/12/13
|
Re: [railo] Re: After Update from jre-7_21 to jre-7_45: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory() failed; error='Cannot allocate memory' (errno=12)
Good find Chris!
https://github.com/utdream/CFML-Installers/issues/57
Warm Regards,
Jordan Michaels
On 11/11/2013 07:28 AM, Chris Blackwell wrote:
> Patrick (and Bruce)
>
> Do not use JAVA_OPTS to pass arguments to tomcat, use CATALINA_OPTS
>
> for an explanation of why, see ->
> http://www.wellho.net/mouth/2163_CATALINA-OPTS-v-JAVA-OPTS-What-is-the-difference-.html
>
> basically, if you set -Xms1024m in JAVA_OPTS the java process that is
> spawned to restart tomcat also uses that amount of memory, so only use
> global options in JAVA_OPTS, not per process stuff which for tomcat
> should be placed in CATALINA_OPTS.
>
> Chris
>
>
> On 11 November 2013 12:05, Patrick Heppler < slash...@googlemail.com
>>> -Dfile.encoding=UTF-8 -Duser.language=de -Duser.region=DE
>>> -Duser.timezone=Europe/Berlin
>>> -Dcoldfusion.jsafe.__defaultalgo=FIPS186Random
>> < https://groups.google.com/d/msgid/railo/585C7698-F6A1-4422-9DA1-D26019A8AE9D%40gmail.com>.
>> For more options, visit
>> https://groups.google.com/__groups/opt_out
>> < https://groups.google.com/groups/opt_out>.
https://github.com/utdream/CFML-Installers/issues/57
Warm Regards,
Jordan Michaels
On 11/11/2013 07:28 AM, Chris Blackwell wrote:
> Patrick (and Bruce)
>
> Do not use JAVA_OPTS to pass arguments to tomcat, use CATALINA_OPTS
>
> for an explanation of why, see ->
> http://www.wellho.net/mouth/2163_CATALINA-OPTS-v-JAVA-OPTS-What-is-the-difference-.html
>
> basically, if you set -Xms1024m in JAVA_OPTS the java process that is
> spawned to restart tomcat also uses that amount of memory, so only use
> global options in JAVA_OPTS, not per process stuff which for tomcat
> should be placed in CATALINA_OPTS.
>
> Chris
>
>
> On 11 November 2013 12:05, Patrick Heppler < slash...@googlemail.com
> <mailto:
slash...@googlemail.com>> wrote:
>
> I now switched back from custom jetty setup to railo/tomcat
> (installer bundle) because I had an issue with javaagent.
>
> I now have this Java Opts:
> JAVA_OPTS="-server -Djava.io.tmpdir=/dev/shm
> -javaagent:/usr/share/nginx/html/ example.com/WEB-INF/lib/railo-inst.jar
> <
http://example.com/WEB-INF/lib/railo-inst.jar>
>
> I now switched back from custom jetty setup to railo/tomcat
> (installer bundle) because I had an issue with javaagent.
>
> I now have this Java Opts:
> JAVA_OPTS="-server -Djava.io.tmpdir=/dev/shm
> -javaagent:/usr/share/nginx/html/ example.com/WEB-INF/lib/railo-inst.jar
> -Dsun.io.useCanonCaches=false -Xms1024m -Xmx2048m
> -XX:MaxPermSize=256M -XX:+HeapDumpOnOutOfMemoryError
> -Djava.security.egd=/dev/urandom -Dfile.encoding=UTF-8
> -Duser.language=de -Duser.region=DE -Duser.timezone=Europe/Berlin
> -Dcoldfusion.jsafe.defaultalgo=FIPS186Random";
>
> I still get the memory errors, not on railo start but on restart.
> Linux top command shows more then 1.4 GB free while railo is running.
> Railo Server Admin shows 30% Heap and 20% Non-Heap used.
>
>
> Am Montag, 4. November 2013 20:40:06 UTC+1 schrieb Igal:
>
>> This is where I got this settings from
> then I suggest that Bruce update his blog post ;)
>
>
> On 11/4/2013 11:38 AM, Kai Koenig wrote:
>> That's fine, but be aware and warned that JVM/GC settings are
>> always heavily depending on one's hardware, OS, configuration,
>> application as well as _optimisation goals_.
>>
>> I strongly advise against copying any sort of JVM/GC settings
>> from anyone's blog or tutorial. They might really well work
>> for Bruce's setup but they might (and apparently do) cause
>> issue for you.
>>
>> Cheers
>> Kai
>>
>>> Just FYI
>>>
https://www.farbeyondcode.__com/Optimizing-Tomcat-s-JVM-__Options-for-Railo-4-1-5-3001.__html
> -XX:MaxPermSize=256M -XX:+HeapDumpOnOutOfMemoryError
> -Djava.security.egd=/dev/urandom -Dfile.encoding=UTF-8
> -Duser.language=de -Duser.region=DE -Duser.timezone=Europe/Berlin
> -Dcoldfusion.jsafe.defaultalgo=FIPS186Random";
>
> I still get the memory errors, not on railo start but on restart.
> Linux top command shows more then 1.4 GB free while railo is running.
> Railo Server Admin shows 30% Heap and 20% Non-Heap used.
>
>
> Am Montag, 4. November 2013 20:40:06 UTC+1 schrieb Igal:
>
>> This is where I got this settings from
> then I suggest that Bruce update his blog post ;)
>
>
> On 11/4/2013 11:38 AM, Kai Koenig wrote:
>> That's fine, but be aware and warned that JVM/GC settings are
>> always heavily depending on one's hardware, OS, configuration,
>> application as well as _optimisation goals_.
>>
>> I strongly advise against copying any sort of JVM/GC settings
>> from anyone's blog or tutorial. They might really well work
>> for Bruce's setup but they might (and apparently do) cause
>> issue for you.
>>
>> Cheers
>> Kai
>>
>>> Just FYI
>>> <
https://www.farbeyondcode.com/Optimizing-Tomcat-s-JVM-Options-for-Railo-4-1-5-3001.html>
>>>
>>> This is where I got this settings from. But I'll try the JVM
>>> Settings recommended
>>>
>>> Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick
>>> Heppler:
>>>
>>> Hi,
>>> I'm running railo 4.1.1.009 on a centos box. Today I
>>> updated JRE (Oracle) from jre-7_21 to jre-7_45. And now
>>> when starting jetty I get:
>>> Starting Jetty: Java HotSpot(TM) 64-Bit Server VM
>>> warning: INFO: os::commit_memory(__0x00000007e0000000,
>>>
>>> This is where I got this settings from. But I'll try the JVM
>>> Settings recommended
>>>
>>> Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick
>>> Heppler:
>>>
>>> Hi,
>>> I'm running railo 4.1.1.009 on a centos box. Today I
>>> updated JRE (Oracle) from jre-7_21 to jre-7_45. And now
>>> when starting jetty I get:
>>> Starting Jetty: Java HotSpot(TM) 64-Bit Server VM
>>> 536870912, 2097152, 0) failed; error='Cannot allocate
>>> memory' (errno=12)
>>>
>>> With jre-7_21 I had no issue. Railo is running fine, anyway.
>>>
>>> My Java Args (from a Railo Blog ;) ):
>>> -server -Djava.io.tmpdir=/dev/shm
>>> -javaagent:/opt/jetty/lib/ext/__railo-inst.jar
>>> memory' (errno=12)
>>>
>>> With jre-7_21 I had no issue. Railo is running fine, anyway.
>>>
>>> My Java Args (from a Railo Blog ;) ):
>>> -server -Djava.io.tmpdir=/dev/shm
>>> -Dsun.io.useCanonCaches=false -Xms1024m -Xmx2048m
>>> -XX:PermSize=512M -XX:MaxPermSize=512M -XX:NewSize=512M
>>> -XX:MaxNewSize=512M -XX:GCTimeRatio=5
>>> -XX:ThreadPriorityPolicy=42 -XX:ParallelGCThreads=4
>>> -XX:+UseParNewGC -XX:MaxGCPauseMillis=50
>>> -XX:+DisableExplicitGC -XX:MaxHeapFreeRatio=70
>>> -XX:MinHeapFreeRatio=40 -XX:+UseStringCache
>>> -XX:+OptimizeStringConcat -XX:+UseTLAB
>>> -XX:+ScavengeBeforeFullGC -XX:CompileThreshold=1500
>>> -XX:+TieredCompilation -XX:+UseBiasedLocking
>>> -Xverify:none -XX:+UseThreadPriorities -XX:+UseLargePages
>>> -XX:LargePageSizeInBytes=2m -XX:+UseFastAccessorMethods
>>> -XX:+UseCompressedOops -Djava.security.egd=/dev/__urandom
>>> -XX:PermSize=512M -XX:MaxPermSize=512M -XX:NewSize=512M
>>> -XX:MaxNewSize=512M -XX:GCTimeRatio=5
>>> -XX:ThreadPriorityPolicy=42 -XX:ParallelGCThreads=4
>>> -XX:+UseParNewGC -XX:MaxGCPauseMillis=50
>>> -XX:+DisableExplicitGC -XX:MaxHeapFreeRatio=70
>>> -XX:MinHeapFreeRatio=40 -XX:+UseStringCache
>>> -XX:+OptimizeStringConcat -XX:+UseTLAB
>>> -XX:+ScavengeBeforeFullGC -XX:CompileThreshold=1500
>>> -XX:+TieredCompilation -XX:+UseBiasedLocking
>>> -Xverify:none -XX:+UseThreadPriorities -XX:+UseLargePages
>>> -XX:LargePageSizeInBytes=2m -XX:+UseFastAccessorMethods
>>> -Dfile.encoding=UTF-8 -Duser.language=de -Duser.region=DE
>>> -Duser.timezone=Europe/Berlin
>>> -Dcoldfusion.jsafe.__defaultalgo=FIPS186Random
>>>
>>
>>
>> --
>> Did you find this reply useful? Help the Railo community and
>> add it to the Railo Server wiki at
>>
https://github.com/getrailo/__railo/wiki
>>
>>
>> --
>> Did you find this reply useful? Help the Railo community and
>> add it to the Railo Server wiki at
>> <
https://github.com/getrailo/railo/wiki>
>> ---
>> You received this message because you are subscribed to the
>> Google Groups "Railo" group.
>> To view this discussion on the web visit
>>
https://groups.google.com/d/__msgid/railo/585C7698-F6A1-__4422-9DA1-D26019A8AE9D%__40gmail.com
>> ---
>> You received this message because you are subscribed to the
>> Google Groups "Railo" group.
>> To view this discussion on the web visit
>> < https://groups.google.com/d/msgid/railo/585C7698-F6A1-4422-9DA1-D26019A8AE9D%40gmail.com>.
>> For more options, visit
>> https://groups.google.com/__groups/opt_out
>> < https://groups.google.com/groups/opt_out>.
>
> --
> Igal Sapir
> Railo Core Developer
> http://getRailo.org/
>
> --
> Did you find this reply useful? Help the Railo community and add it
> to the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/railo/39c735b9-e2ee-4486-8647-ee7f25621066%40googlegroups.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> Igal Sapir
> Railo Core Developer
> http://getRailo.org/
>
> --
> Did you find this reply useful? Help the Railo community and add it
> to the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/railo/39c735b9-e2ee-4486-8647-ee7f25621066%40googlegroups.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit
>
https://groups.google.com/d/msgid/railo/CAB%3DtfToJGsJU%2BC5PiVU%3DJ6mq5Yz9voDkEAuuG7Tpy0_%2B6bSKyg%40mail.gmail.com.
> Did you find this reply useful? Help the Railo community and add it to
> the Railo Server wiki at https://github.com/getrailo/railo/wiki
> ---
> You received this message because you are subscribed to the Google
> Groups "Railo" group.
> To view this discussion on the web visit
- show quoted text -
I made some tweaks to the blog article now, but I still prefer the non-default options I use. Some of them probably have no effect, others have a real effect on garbage collection timings.
My app runs a bunch of CMS sites that have very little write activity. OS / Hardware is pretty typical. Works good still.
- show quoted text -
|
|
Patrick Heppler |
11/12/13
|
Changed JAVA_OPTS to CATALINA_OPTS, works fine.
Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick Heppler:
Am Montag, 28. Oktober 2013 12:53:35 UTC+1 schrieb Patrick Heppler:
Hi,- show quoted text -
|
|
Bruce Kirkpatrick |
11/12/13
|
If that was the problem, I'd guess the system has not much memory available to the rest of the system. You might want to leave around 25% of the memory unused so linux can cache stuff and avoid swapping. I usually give Railo less then 1/3 of the total memory so database and other services have a 1/3 and system has a third. I'd give database a greater share in a larger system.
- show quoted text -