Commit 77cebd6b authored by mhahnenberg@apple.com's avatar mhahnenberg@apple.com

run-jsc-stress-tests only supports host environments that have make installed

https://bugs.webkit.org/show_bug.cgi?id=124550

Reviewed by Darin Adler.

This might not be the case for all hosts, so this patch implements an alternate "backend" 
for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at 
least somewhat competitive with the make-based test runner, the shell backend uses subshells
run in the background to allow tests to run in parallel. Since the concurrency primitives 
in shell scripting are rather coarse, the overhead of this parallelism is higher than that 
of the make-based runner.

* Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into
the bundle and controls all of the parallel aspects of the shell-based test runner.
* Scripts/run-jsc-stress-tests:


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@160016 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent 134cb181
2013-12-03 Mark Hahnenberg <mhahnenberg@apple.com>
run-jsc-stress-tests only supports host environments that have make installed
https://bugs.webkit.org/show_bug.cgi?id=124550
Reviewed by Darin Adler.
This might not be the case for all hosts, so this patch implements an alternate "backend"
for run-jsc-stress-tests to use normal shell commands rather than Makefiles. To remain at
least somewhat competitive with the make-based test runner, the shell backend uses subshells
run in the background to allow tests to run in parallel. Since the concurrency primitives
in shell scripting are rather coarse, the overhead of this parallelism is higher than that
of the make-based runner.
* Scripts/jsc-stress-test-helpers/shell-runner.sh: Added. This is the runner that is copied into
the bundle and controls all of the parallel aspects of the shell-based test runner.
* Scripts/run-jsc-stress-tests:
2013-12-03 Tamas Gergely <tgergely.u-szeged@partner.samsung.com>
Remove function from TextChecker
......
# Copyright (C) 2013 Apple Inc. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
#
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
numProcs=`sysctl -n hw.activecpu`
if [ $? -gt 0 ]
then
numProcs=`nproc --all 2>/dev/null`
if [ $? -gt 0 ]
then
numProcs=1
fi
fi
indexFile=".index"
testList=".all_tests.txt"
tempFile=".temp.txt"
lockDir=".lock_dir"
trap "kill -9 0" SIGINT
echo 0 > ${indexFile}
ls test_script_* > ${testList}
function lock_test_list() {
until mkdir ${lockDir} 2> /dev/null; do sleep 0; done
}
function unlock_test_list() {
rmdir ${lockDir}
}
total=`wc -l < "${testList}" | sed 's/ //g'`
for proc in `seq ${numProcs}`
do
(
lock_test_list
while [ -s ${testList} ]
do
index=`cat ${indexFile}`
index=$((index + 1))
echo "${index}" > ${indexFile}
printf "\r ${index}/${total}"
nextTest=`tail -n 1 ${testList}`
sed '$d' < ${testList} > ${tempFile}
mv ${tempFile} ${testList}
unlock_test_list
sh ${nextTest} > /dev/null
lock_test_list
done
unlock_test_list
)&
done
wait
......@@ -68,7 +68,7 @@ def printCommandArray(*cmd)
end
def mysys(*cmd)
printCommandArray(*cmd)
printCommandArray(*cmd) if $verbosity >= 1
raise "Command failed: #{$?.inspect}" unless system(*cmd)
end
......@@ -92,6 +92,7 @@ $verbosity = 0
$bundle = nil
$tarball = false
$copyVM = false
$testRunnerType = :make
def usage
puts "run-jsc-stress-tests -j <shell path> <collections path> [<collections path> ...]"
......@@ -102,6 +103,8 @@ def usage
puts "--verbose (-v) Print more things while running."
puts "--run-bundle Runs a bundle previously created by run-jsc-stress-tests."
puts "--tarball Creates a tarball of the final bundle."
puts "--shell-runner Uses the shell-based test runner instead of the default make-based runner."
puts " In general the shell runner is slower than the make runner."
puts "--help (-h) Print this message."
exit 1
end
......@@ -113,6 +116,7 @@ GetoptLong.new(['--help', '-h', GetoptLong::NO_ARGUMENT],
['--run-bundle', GetoptLong::REQUIRED_ARGUMENT],
['--tarball', GetoptLong::NO_ARGUMENT],
['--force-vm-copy', GetoptLong::NO_ARGUMENT],
['--shell-runner', GetoptLong::NO_ARGUMENT],
['--verbose', '-v', GetoptLong::NO_ARGUMENT]).each {
| opt, arg |
case opt
......@@ -133,6 +137,8 @@ GetoptLong.new(['--help', '-h', GetoptLong::NO_ARGUMENT],
$copyVM = true
when '--force-vm-copy'
$copyVM = true
when '--shell-runner'
$testRunnerType = :shell
end
}
......@@ -381,7 +387,7 @@ class Plan
end
def reproScriptCommand
# We have to find our way back to the .parallel directory since that's where all of the relative
# We have to find our way back to the .runner directory since that's where all of the relative
# paths assume they start out from.
script = "CURRENT_DIR=\"$( cd \"$( dirname \"${BASH_SOURCE[0]}\" )\" && pwd )\"\n"
script += "cd $CURRENT_DIR\n"
......@@ -389,7 +395,7 @@ class Plan
| pathComponent |
script += "cd ..\n"
}
script += "cd .parallel\n"
script += "cd .runner\n"
script += "export DYLD_FRAMEWORK_PATH=$(cd ../#{$frameworkPath.dirname}; pwd)\n"
IMPORTANT_ENVS.each {
......@@ -916,13 +922,54 @@ def eachResultFile(startingDir, &block)
end
end
def prepareParallelTestRunner
def prepareTestRunner
raise if $bundle
$runlist.each_with_index {
| plan, index |
plan.index = index
}
Dir.mkdir($runnerDir) unless $runnerDir.directory?
toDelete = []
Dir.foreach($runnerDir) {
| filename |
if filename =~ /^test_/
toDelete << filename
end
}
toDelete.each {
| filename |
File.unlink($runnerDir + filename)
}
$runlist.each {
| plan |
plan.writeRunScript($runnerDir + "test_script_#{plan.index}")
}
case $testRunnerType
when :make
prepareMakeTestRunner
when :shell
prepareShellTestRunner
else
raise "Unknown test runner type: #{$testRunnerType.to_s}"
end
end
def prepareShellTestRunner
FileUtils.cp SCRIPTS_PATH + "jsc-stress-test-helpers" + "shell-runner.sh", $runnerDir + "runscript"
end
def prepareMakeTestRunner
# The goals of our parallel test runner are scalability and simplicity. The
# simplicity part is particularly important. We don't want to have to have
# a full-time contributor just philosophising about parallel testing.
#
# As such, we just pass off all of the hard work to 'make'. This creates a
# dummy directory ("$outputDir/.parallel") in which we create a dummy
# dummy directory ("$outputDir/.runner") in which we create a dummy
# Makefile. The Makefile has an 'all' rule that depends on all of the tests.
# That is, for each test we know we will run, there is a rule in the
# Makefile and 'all' depends on it. Running 'make -j <whatever>' on this
......@@ -945,40 +992,19 @@ def prepareParallelTestRunner
# continue past that failure and complete all the tests anyway.
#
# In the end, this script collects all of the failures by searching for
# files in the .parallel directory whose name matches /^test_fail_/, where
# files in the .runner directory whose name matches /^test_fail_/, where
# the thing after the 'fail_' is the test index. Those are the files that
# would be created by the test scripts if they detect failure. We're
# basically using the filesystem as a concurrent database of test failures.
# Even if two tests fail at the same time, since they're touching different
# files we won't miss any failures.
runIndices = []
$runlist.each_with_index {
| plan, index |
runIndices << index
plan.index = index
}
Dir.mkdir($parallelDir) unless $parallelDir.directory?
toDelete = []
Dir.foreach($parallelDir) {
| filename |
if filename =~ /^test_/
toDelete << filename
end
}
toDelete.each {
| filename |
File.unlink($parallelDir + filename)
}
$runlist.each {
| plan |
plan.writeRunScript($parallelDir + "test_script_#{plan.index}")
runIndices << plan.index
}
File.open($parallelDir + "Makefile", "w") {
File.open($runnerDir + "Makefile", "w") {
| outp |
outp.puts("all: " + runIndices.map{|v| "test_done_#{v}"}.join(' '))
runIndices.each {
......@@ -1004,17 +1030,34 @@ end
puts
def cleanParallelDirectory
def cleanRunnerDirectory
raise unless $bundle
Dir.foreach($parallelDir) {
Dir.foreach($runnerDir) {
| filename |
next unless filename =~ /^test_fail/
FileUtils.rm_f $parallelDir + filename
FileUtils.rm_f $runnerDir + filename
}
end
def runParallelTestRunner
Dir.chdir($parallelDir) {
def runTestRunner
case $testRunnerType
when :shell
runShellTestRunner
when :make
runMakeTestRunner
else
raise "Unknown test runner type: #{$testRunnerType.to_s}"
end
end
def runShellTestRunner
Dir.chdir($runnerDir) {
mysys("sh", "runscript")
}
end
def runMakeTestRunner
Dir.chdir($runnerDir) {
# -1 for the Makefile, and -2 for '..' and '.'
numberOfTests = Dir.entries(".").count - 3
unless $progressMeter
......@@ -1083,7 +1126,7 @@ end
def detectFailures
raise if $bundle
Dir.foreach($parallelDir) {
Dir.foreach($runnerDir) {
| filename |
next unless filename =~ /test_fail_/
appendFailure($runlist[$~.post_match.to_i])
......@@ -1105,13 +1148,16 @@ rescue
end
$outputDir = $outputDir.realpath
$parallelDir = $outputDir + ".parallel"
$runnerDir = $outputDir + ".runner"
prepareBundle unless $bundle
prepareParallelTestRunner unless $bundle
cleanParallelDirectory if $bundle
puts " "
prepareTestRunner unless $bundle
cleanRunnerDirectory if $bundle
cleanOldResults if $bundle
runParallelTestRunner
runTestRunner
cleanEmptyResultFiles
detectFailures unless $bundle
compressBundle if $tarball
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment