Skip to content

Make bundle install and scripts compatible with JRuby. #1006

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
myronmarston wants to merge 1 commit into
base: myron/remove-ox
Choose a base branch
from
Draft

Make bundle install and scripts compatible with JRuby. #1006

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
.claude
!.claude/skills/

.tool-versions

config/release/bin
config/release/bundle
config/release/.bundle
Expand Down
17 changes: 10 additions & 7 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ source "https://rubygems.org"
group :development do
gem "aws_lambda_ric", "~> 3.1", ">= 3.1.3"
# graphql-c_parser is no longer a hard dependency, but we include it here for faster CI tests
gem "graphql-c_parser", "~> 1.1", ">= 1.1.3"
gem "graphql-c_parser", "~> 1.1", ">= 1.1.3", platforms: :ruby
gem "benchmark-ips", "~> 2.14"
gem "coderay", "~> 1.1", ">= 1.1.3"
gem "factory_bot", "~> 6.5", ">= 6.5.6"
gem "faker", "~> 3.6"

# Pin to a GitHub SHA until Ruby 4.0 support has been released
gem "flatware-rspec", "~> 2.3", ">= 2.3.4", github: "briandunn/flatware", ref: "0403ac1137cc7958fe06db2c0563dfbab0bd24db"
gem "flatware-rspec", "~> 2.3", ">= 2.3.4", github: "briandunn/flatware", ref: "0403ac1137cc7958fe06db2c0563dfbab0bd24db", platforms: :ruby

gem "httpx", "~> 1.7", ">= 1.7.2"
gem "memory_profiler", "~> 1.1"
Expand All @@ -40,7 +40,7 @@ group :development do
gem "simplecov", "~> 0.22"
gem "simplecov-console", "~> 0.9", ">= 0.9.4"
gem "standard", "~> 1.53.0"
gem "steep", "~> 1.10.0"
gem "steep", "~> 1.10.0", platforms: :ruby
gem "super_diff", "~> 0.18"
gem "vcr", "~> 6.4"
end
Expand All @@ -50,9 +50,11 @@ group :site do
# Pin to a GitHub SHA until Ruby 4.0 support has been released: https://github.com/filewatcher/filewatcher/issues/286
gem "filewatcher", "~> 2.1", github: "filewatcher/filewatcher", ref: "596fad65f3b442fee6cf30a3d8daf2767d63f8c9"

gem "html-proofer", "~> 5.2"
gem "jekyll", "~> 4.4", ">= 4.4.1"
gem "redcarpet", "~> 3.6", ">= 3.6.1"
platforms :ruby do
gem "html-proofer", "~> 5.2"
gem "jekyll", "~> 4.4", ">= 4.4.1"
gem "redcarpet", "~> 3.6", ">= 3.6.1"
end

# Pull in a YAML syntax highlighting fix so that our JSON schemas render correctly at the website:
# https://github.com/rouge-ruby/rouge/pull/2156
Expand Down Expand Up @@ -81,8 +83,9 @@ if repo_root == __dir__
require_relative "elasticgraph-support/lib/elastic_graph/version"

# Depend on each ElasticGraph gem in the repo.
# Use absolute paths for JRuby compatibility (JRuby resolves `path:` relative to cwd, not Gemfile)
gems_in_this_repo.map do |name|
gem name, ::ElasticGraph::VERSION, path: name
gem name, ::ElasticGraph::VERSION, path: "#{repo_root}/#{name}"
end
else
# Otherwise, we just load the local `.gemspec` file in the current directory.
Expand Down
38 changes: 38 additions & 0 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ GEM
benchmark (0.5.0)
benchmark-ips (2.14.0)
bigdecimal (4.0.1)
bigdecimal (4.0.1-java)
coderay (1.1.3)
colorator (1.1.0)
concurrent-ruby (1.3.6)
Expand All @@ -300,6 +301,7 @@ GEM
json
csv (3.3.5)
date (3.5.1)
date (3.5.1-java)
diff-lcs (1.6.2)
docile (1.4.1)
drb (2.2.3)
Expand All @@ -315,9 +317,11 @@ GEM
eventmachine (>= 0.12.9)
http_parser.rb (~> 0)
erb (6.0.1)
erb (6.0.1-java)
ethon (0.15.0)
ffi (>= 1.15.0)
eventmachine (1.2.7)
eventmachine (1.2.7-java)
factory_bot (6.5.6)
activesupport (>= 6.1.0)
faker (3.6.0)
Expand All @@ -338,6 +342,10 @@ GEM
ffi (1.17.3-arm-linux-gnu)
ffi (1.17.3-arm-linux-musl)
ffi (1.17.3-arm64-darwin)
ffi (1.17.3-java)
ffi-compiler (1.3.2)
ffi (>= 1.15.5)
rake
fiber-annotation (0.2.0)
fiber-local (1.1.0)
fiber-storage
Expand All @@ -350,6 +358,11 @@ GEM
google-protobuf (4.33.4-arm64-darwin)
bigdecimal
rake (>= 13)
google-protobuf (4.33.4-java)
bigdecimal
ffi (~> 1)
ffi-compiler (~> 1)
rake (>= 13)
graphql (2.5.18)
base64
fiber-storage
Expand All @@ -376,11 +389,13 @@ GEM
i18n (1.14.8)
concurrent-ruby (~> 1.0)
io-console (0.8.2)
io-console (0.8.2-java)
io-event (1.14.2)
irb (1.16.0)
pp (>= 0.6.0)
rdoc (>= 4.0.0)
reline (>= 0.4.2)
jar-dependencies (0.5.5)
jekyll (4.4.1)
addressable (~> 2.4)
base64 (~> 0.2)
Expand All @@ -406,6 +421,7 @@ GEM
listen (~> 3.0)
jmespath (1.6.2)
json (2.18.0)
json (2.18.0-java)
json_schemer (2.5.0)
bigdecimal
hana (~> 1.3)
Expand Down Expand Up @@ -444,6 +460,8 @@ GEM
racc (~> 1.4)
nokogiri (1.19.0-arm64-darwin)
racc (~> 1.4)
nokogiri (1.19.0-java)
racc (~> 1.4)
opensearch-ruby (3.4.0)
faraday (>= 1.0, < 3)
multi_json (>= 1.0)
Expand All @@ -469,8 +487,12 @@ GEM
psych (5.3.1)
date
stringio
psych (5.3.1-java)
date
jar-dependencies (>= 0.1.7)
public_suffix (7.0.2)
racc (1.8.1)
racc (1.8.1-java)
rack (3.2.4)
rack-test (2.2.0)
rack (>= 1.3)
Expand Down Expand Up @@ -602,6 +624,7 @@ GEM
uri (>= 0.12.0)
stringio (3.2.0)
strscan (3.1.7)
strscan (3.1.7-java)
super_diff (0.18.0)
attr_extras (>= 6.2.4)
diff-lcs
Expand Down Expand Up @@ -643,6 +666,7 @@ PLATFORMS
riscv64-linux-gnu
riscv64-linux-musl
ruby
universal-java-24
x86-cygwin
x86-linux-android
x86-mingw-ucrt
Expand Down Expand Up @@ -731,13 +755,15 @@ CHECKSUMS
benchmark (0.5.0) sha256=465df122341aedcb81a2a24b4d3bd19b6c67c1530713fd533f3ff034e419236c
benchmark-ips (2.14.0) sha256=b72bc8a65d525d5906f8cd94270dccf73452ee3257a32b89fbd6684d3e8a9b1d
bigdecimal (4.0.1) sha256=8b07d3d065a9f921c80ceaea7c9d4ae596697295b584c296fe599dd0ad01c4a7
bigdecimal (4.0.1-java) sha256=95c40a0a92dfb83b293f4b9865cd27caa7c045bedab86f3620e5dc27e6553083
coderay (1.1.3) sha256=dc530018a4684512f8f38143cd2a096c9f02a1fc2459edcfe534787a7fc77d4b
colorator (1.1.0) sha256=e2f85daf57af47d740db2a32191d1bdfb0f6503a0dfbc8327d0c9154d5ddfc38
concurrent-ruby (1.3.6) sha256=6b56837e1e7e5292f9864f34b69c5a2cbc75c0cf5338f1ce9903d10fa762d5ab
connection_pool (3.0.2) sha256=33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a
console (1.34.2) sha256=1c036abf606ccec83f9dc28f0c31710fe5936ffe7ba5d235ae2865590a482d58
csv (3.3.5) sha256=6e5134ac3383ef728b7f02725d9872934f523cb40b961479f69cf3afa6c8e73f
date (3.5.1) sha256=750d06384d7b9c15d562c76291407d89e368dda4d4fff957eb94962d325a0dc0
date (3.5.1-java) sha256=12e09477dc932afe45bf768cd362bf73026804e0db1e6c314186d6cd0bee3344
diff-lcs (1.6.2) sha256=9ae0d2cba7d4df3075fe8cd8602a8604993efc0dfa934cff568969efb1909962
docile (1.4.1) sha256=96159be799bfa73cdb721b840e9802126e4e03dfc26863db73647204c727f21e
drb (2.2.3) sha256=0b00d6fdb50995fe4a45dea13663493c841112e4068656854646f418fda13373
Expand Down Expand Up @@ -770,8 +796,10 @@ CHECKSUMS
elasticsearch-api (9.2.0) sha256=775136055d56a3d23be13a93ba8d67c5bb0ed2895f15270ca144fbc0ae920190
em-websocket (0.5.3) sha256=f56a92bde4e6cb879256d58ee31f124181f68f8887bd14d53d5d9a292758c6a8
erb (6.0.1) sha256=28ecdd99c5472aebd5674d6061e3c6b0a45c049578b071e5a52c2a7f13c197e5
erb (6.0.1-java) sha256=5c6b8d885fb0220d4a8ad158f70430d805845939dd44827e5130ef7fdbaed8ba
ethon (0.15.0) sha256=0809805a035bc10f54162ca99f15ded49e428e0488bcfe1c08c821e18261a74d
eventmachine (1.2.7) sha256=994016e42aa041477ba9cff45cbe50de2047f25dd418eba003e84f0d16560972
eventmachine (1.2.7-java) sha256=11ab7af3505f53ed7565d6a13fe73568fe0d6fac647c8816430e9c3171c8c58f
factory_bot (6.5.6) sha256=12beb373214dccc086a7a63763d6718c49769d5606f0501e0a4442676917e077
faker (3.6.0) sha256=4ce80bf91c8d09bbfff4c5596690bf862d60eac420f86737ca8ce12a54dc464a
faraday (2.14.0) sha256=8699cfe5d97e55268f2596f9a9d5a43736808a943714e3d9a53e6110593941cd
Expand All @@ -783,6 +811,8 @@ CHECKSUMS
ffi (1.17.3-arm-linux-gnu) sha256=5bd4cea83b68b5ec0037f99c57d5ce2dd5aa438f35decc5ef68a7d085c785668
ffi (1.17.3-arm-linux-musl) sha256=0d7626bb96265f9af78afa33e267d71cfef9d9a8eb8f5525344f8da6c7d76053
ffi (1.17.3-arm64-darwin) sha256=0c690555d4cee17a7f07c04d59df39b2fba74ec440b19da1f685c6579bb0717f
ffi (1.17.3-java) sha256=be1ecc86edbf71b9ab59212a23b33efec431c0446cb3183066d8a960a1946d3a
ffi-compiler (1.3.2) sha256=a94f3d81d12caf5c5d4ecf13980a70d0aeaa72268f3b9cc13358bcc6509184a0
fiber-annotation (0.2.0) sha256=7abfadf1d119f508867d4103bf231c0354d019cc39a5738945dec2edadaf6c03
fiber-local (1.1.0) sha256=c885f94f210fb9b05737de65d511136ea602e00c5105953748aa0f8793489f06
fiber-storage (1.0.1) sha256=f48e5b6d8b0be96dac486332b55cee82240057065dc761c1ea692b2e719240e1
Expand All @@ -793,6 +823,7 @@ CHECKSUMS
forwardable-extended (2.6.0) sha256=1bec948c469bbddfadeb3bd90eb8c85f6e627a412a3e852acfd7eaedbac3ec97
google-protobuf (4.33.4) sha256=86921935b023ed0d872d6e84382e79016c91689be0520d614c74426778f13c16
google-protobuf (4.33.4-arm64-darwin) sha256=63bb57e3d7108986f961546a536adaa38bd9ab5b1ebc39a1741a16c6eba3e869
google-protobuf (4.33.4-java) sha256=f0a4c6c93d857f3b86c0419a30e956d6e10abcc115a2ac278047052ccd11886e
graphql (2.5.18) sha256=857dc1945e014932077eb7aedf4d4c1b086810faa3c829ffdbd74813ee294120
graphql-c_parser (1.1.3) sha256=a0a7e3221c091285ca03987de50ba8ad49846f66748b5fdd3780729482bbc358
hana (1.3.7) sha256=5425db42d651fea08859811c29d20446f16af196308162894db208cac5ce9b0d
Expand All @@ -804,13 +835,16 @@ CHECKSUMS
httpx (1.7.2) sha256=10cdcd5b03794de743bec87e8cdba2ca93e2124084b6cdb1ad01b00c192a51f7
i18n (1.14.8) sha256=285778639134865c5e0f6269e0b818256017e8cde89993fdfcbfb64d088824a5
io-console (0.8.2) sha256=d6e3ae7a7cc7574f4b8893b4fca2162e57a825b223a177b7afa236c5ef9814cc
io-console (0.8.2-java) sha256=837efefe96084c13ae91114917986ae6c6d1cf063b27b8419cc564a722a38af8
io-event (1.14.2) sha256=b0a069190eafe86005c22f7464f744971b5bd82f153740d34e6ab49548d4f613
irb (1.16.0) sha256=2abe56c9ac947cdcb2f150572904ba798c1e93c890c256f8429981a7675b0806
jar-dependencies (0.5.5) sha256=2972b9fcba4b014e6446a84b5c09674a3e8648b95b71768e729f0e8e40568059
jekyll (4.4.1) sha256=4c1144d857a5b2b80d45b8cf5138289579a9f8136aadfa6dd684b31fe2bc18c1
jekyll-sass-converter (3.1.0) sha256=83925d84f1d134410c11d0c6643b0093e82e3a3cf127e90757a85294a3862443
jekyll-watch (2.2.1) sha256=bc44ed43f5e0a552836245a54dbff3ea7421ecc2856707e8a1ee203a8387a7e1
jmespath (1.6.2) sha256=238d774a58723d6c090494c8879b5e9918c19485f7e840f2c1c7532cf84ebcb1
json (2.18.0) sha256=b10506aee4183f5cf49e0efc48073d7b75843ce3782c68dbeb763351c08fd505
json (2.18.0-java) sha256=74706f684baeb1a40351ed26fc8fe6e958afa861320d1c28ff4eb7073b29c7aa
json_schemer (2.5.0) sha256=2f01fb4cce721a4e08dd068fc2030cffd0702a7f333f1ea2be6e8991f00ae396
kramdown (2.5.2) sha256=1ba542204c66b6f9111ff00dcc26075b95b220b07f2905d8261740c82f7f02fa
kramdown-parser-gfm (1.1.0) sha256=fb39745516427d2988543bf01fc4cf0ab1149476382393e0e9c48592f6581729
Expand All @@ -833,6 +867,7 @@ CHECKSUMS
nokogiri (1.19.0-arm-linux-gnu) sha256=572a259026b2c8b7c161fdb6469fa2d0edd2b61cd599db4bbda93289abefbfe5
nokogiri (1.19.0-arm-linux-musl) sha256=23ed90922f1a38aed555d3de4d058e90850c731c5b756d191b3dc8055948e73c
nokogiri (1.19.0-arm64-darwin) sha256=0811dfd936d5f6dd3f6d32ef790568bf29b2b7bead9ba68866847b33c9cf5810
nokogiri (1.19.0-java) sha256=5f3a70e252be641d8a4099f7fb4cc25c81c632cb594eec9b4b8f2ca8be4374f3
opensearch-ruby (3.4.0) sha256=0a8621686bed3c59b4c23e08cbaef873685a3fe4568e9d2703155ca92b8ca05d
optimist (3.2.1) sha256=8cf8a0fd69f3aa24ab48885d3a666717c27bc3d9edd6e976e18b9d771e72e34e
parallel (1.27.0) sha256=4ac151e1806b755fb4e2dc2332cbf0e54f2e24ba821ff2d3dcf86bf6dc4ae130
Expand All @@ -844,8 +879,10 @@ CHECKSUMS
prettyprint (0.2.0) sha256=2bc9e15581a94742064a3cc8b0fb9d45aae3d03a1baa6ef80922627a0766f193
prism (1.8.0) sha256=84453a16ef5530ea62c5f03ec16b52a459575ad4e7b9c2b360fd8ce2c39c1254
psych (5.3.1) sha256=eb7a57cef10c9d70173ff74e739d843ac3b2c019a003de48447b2963d81b1974
psych (5.3.1-java) sha256=20a4a81ad01479ef060f604ed75ba42fe673169e67d923b1bae5aa4e13cc5820
public_suffix (7.0.2) sha256=9114090c8e4e7135c1fd0e7acfea33afaab38101884320c65aaa0ffb8e26a857
racc (1.8.1) sha256=4a7f6929691dbec8b5209a0b373bc2614882b55fc5d2e447a21aaa691303d62f
racc (1.8.1-java) sha256=54f2e6d1e1b91c154013277d986f52a90e5ececbe91465d29172e49342732b98
rack (3.2.4) sha256=5d74b6f75082a643f43c1e76b419c40f0e5527fcfee1e669ac1e6b73c0ccb6f6
rack-test (2.2.0) sha256=005a36692c306ac0b4a9350355ee080fd09ddef1148a5f8b2ac636c720f5c463
rackup (2.3.1) sha256=6c79c26753778e90983761d677a48937ee3192b3ffef6bc963c0950f94688868
Expand Down Expand Up @@ -898,6 +935,7 @@ CHECKSUMS
steep (1.10.0) sha256=1b295b55f9aaff1b8d3ee42453ee55bc2a1078fda0268f288edb2dc014f4d7d1
stringio (3.2.0) sha256=c37cb2e58b4ffbd33fe5cd948c05934af997b36e0b6ca6fdf43afa234cf222e1
strscan (3.1.7) sha256=5f76462b94a3ea50b44973225b7d75b2cb96d4e1bee9ef1319b99ca117b72c8c
strscan (3.1.7-java) sha256=acde34c12323095e46f9c46d869d41d0170e0a6817fc723ffc296372ed1088ee
super_diff (0.18.0) sha256=9f5e77464fa75150619f7783174fbbe1bbac50a1eaf157cd39ad5584b0eac142
terminal-table (3.0.2) sha256=f951b6af5f3e00203fb290a669e0a85c5dd5b051b3b023392ccfd67ba5abae91
thor (1.5.0) sha256=e3a9e55fe857e44859ce104a84675ab6e8cd59c650a49106a05f55f136425e73
Expand Down
34 changes: 20 additions & 14 deletions script/flatware_rspec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,25 @@
# Verbose form of `set -x`.
set -o xtrace

# Avoid a connection issue if a prior run was interrupted. More info:
# https://github.com/briandunn/flatware/issues/68
rm -f flatware-sink
# Check if flatware is available (not on JRuby due to usage of `fork`)
if bundle show flatware 2>/dev/null; then
# Avoid a connection issue if a prior run was interrupted. More info:
# https://github.com/briandunn/flatware/issues/68
rm -f flatware-sink

# Limit the number of workers to 8. On some really beefy CI worker hosts, we've seen
# flatware default to 96 workers (due to the host having 96 vCPUs!), but that's overkill,
# and using too many workers can perform worse.
#
# In addition, using too many parallel processes can overwhelm the locally booted datastore,
# because we have to create a separate set of indices for each process. The datastore tends
# to run into limits when we use 16 processes. 8 processes seems to work better
#
# Note: `Etc.nprocessors` is what flatware uses internally for the default number of workers.
worker_count=$(ruby -retc -e "puts [8, Etc.nprocessors].min")
# Limit the number of workers to 8. On some really beefy CI worker hosts, we've seen
# flatware default to 96 workers (due to the host having 96 vCPUs!), but that's overkill,
# and using too many workers can perform worse.
#
# In addition, using too many parallel processes can overwhelm the locally booted datastore,
# because we have to create a separate set of indices for each process. The datastore tends
# to run into limits when we use 16 processes. 8 processes seems to work better
#
# Note: `Etc.nprocessors` is what flatware uses internally for the default number of workers.
worker_count=$(ruby -retc -e "puts [8, Etc.nprocessors].min")

bundle exec flatware rspec -w $worker_count "$@"
bundle exec flatware rspec -w $worker_count "$@"
else
# Fall back to regular rspec when flatware unavailable (e.g. on JRuby)
bundle exec rspec "$@"
fi
Loading