Skip to content

Commit 3bf43fb

Browse files
committed
[SPARK-29159][BUILD] Increase ReservedCodeCacheSize to 1G
### What changes were proposed in this pull request? This PR aims to increase the JVM CodeCacheSize from 0.5G to 1G. ### Why are the changes needed? After upgrading to `Scala 2.12.10`, the following is observed during building. ``` 2019-09-18T20:49:23.5030586Z OpenJDK 64-Bit Server VM warning: CodeCache is full. Compiler has been disabled. 2019-09-18T20:49:23.5032920Z OpenJDK 64-Bit Server VM warning: Try increasing the code cache size using -XX:ReservedCodeCacheSize= 2019-09-18T20:49:23.5034959Z CodeCache: size=524288Kb used=521399Kb max_used=521423Kb free=2888Kb 2019-09-18T20:49:23.5035472Z bounds [0x00007fa62c000000, 0x00007fa64c000000, 0x00007fa64c000000] 2019-09-18T20:49:23.5035781Z total_blobs=156549 nmethods=155863 adapters=592 2019-09-18T20:49:23.5036090Z compilation: disabled (not enough contiguous free space left) ``` ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Manually check the Jenkins or GitHub Action build log (which should not have the above). Closes apache#25836 from dongjoon-hyun/SPARK-CODE-CACHE-1G. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent b917a65 commit 3bf43fb

File tree

7 files changed

+8
-8
lines changed

7 files changed

+8
-8
lines changed

.github/workflows/master.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,6 @@ jobs:
2929
version: ${{ matrix.java }}
3030
- name: Build with Maven
3131
run: |
32-
export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
32+
export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
3333
export MAVEN_CLI_OPTS="--no-transfer-progress"
3434
./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -P${{ matrix.hadoop }} -Phadoop-cloud -Djava.version=${{ matrix.java }} package

build/mvn

+1-1
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ _DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
2222
# Preserve the calling directory
2323
_CALLING_DIR="$(pwd)"
2424
# Options used during compilation
25-
_COMPILE_JVM_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m"
25+
_COMPILE_JVM_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g"
2626

2727
# Installs any application tarball given a URL, the expected tarball name,
2828
# and, optionally, a checkable binary path to determine if the binary has

dev/appveyor-install-dependencies.ps1

+1-1
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@ Invoke-Expression "7z.exe x maven.zip"
9090
# add maven to environment variables
9191
$env:PATH = "$tools\apache-maven-$mavenVer\bin;" + $env:PATH
9292
$env:M2_HOME = "$tools\apache-maven-$mavenVer"
93-
$env:MAVEN_OPTS = "-Xmx2g -XX:ReservedCodeCacheSize=512m"
93+
$env:MAVEN_OPTS = "-Xmx2g -XX:ReservedCodeCacheSize=1g"
9494

9595
Pop-Location
9696

dev/make-distribution.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -160,7 +160,7 @@ fi
160160
# Build uber fat JAR
161161
cd "$SPARK_HOME"
162162

163-
export MAVEN_OPTS="${MAVEN_OPTS:--Xmx2g -XX:ReservedCodeCacheSize=512m}"
163+
export MAVEN_OPTS="${MAVEN_OPTS:--Xmx2g -XX:ReservedCodeCacheSize=1g}"
164164

165165
# Store the command as an array because $MVN variable might have spaces in it.
166166
# Normal quoting tricks don't work.

docs/building-spark.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Spark requires Scala 2.12; support for Scala 2.11 was removed in Spark 3.0.0.
3434

3535
You'll need to configure Maven to use more memory than usual by setting `MAVEN_OPTS`:
3636

37-
export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m"
37+
export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g"
3838

3939
(The `ReservedCodeCacheSize` setting is optional but recommended.)
4040
If you don't add these parameters to `MAVEN_OPTS`, you may see errors and warnings like the following:
@@ -160,7 +160,7 @@ prompt.
160160
Configure the JVM options for SBT in `.jvmopts` at the project root, for example:
161161

162162
-Xmx2g
163-
-XX:ReservedCodeCacheSize=512m
163+
-XX:ReservedCodeCacheSize=1g
164164

165165
For the meanings of these two options, please carefully read the [Setting up Maven's Memory Usage section](https://spark.apache.org/docs/latest/building-spark.html#setting-up-mavens-memory-usage).
166166

pom.xml

+1-1
Original file line numberDiff line numberDiff line change
@@ -240,7 +240,7 @@
240240
-->
241241
<spark.test.home>${session.executionRootDirectory}</spark.test.home>
242242

243-
<CodeCacheSize>512m</CodeCacheSize>
243+
<CodeCacheSize>1g</CodeCacheSize>
244244
</properties>
245245
<repositories>
246246
<repository>

resource-managers/kubernetes/integration-tests/pom.xml

+1-1
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@
141141
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
142142
<junitxml>.</junitxml>
143143
<filereports>SparkTestSuite.txt</filereports>
144-
<argLine>-ea -Xmx4g -XX:ReservedCodeCacheSize=512m ${extraScalaTestArgs}</argLine>
144+
<argLine>-ea -Xmx4g -XX:ReservedCodeCacheSize=1g ${extraScalaTestArgs}</argLine>
145145
<stderr/>
146146
<systemProperties>
147147
<log4j.configuration>file:src/test/resources/log4j.properties</log4j.configuration>

0 commit comments

Comments
 (0)