Skip to content

Memory usage regression since 5/1 #50389

Closed
Closed
Memory usage regression since 5/1#50389
Labels
I-compilememIssue: Problems and improvements with respect to memory usage during compilation.P-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.
Milestone
1.27

Description

@ishitatsuyuki
ishitatsuyuki
opened on May 2, 2018

It's back, caused by presumably #50197.

cc @nikomatsakis

   Compiling syntax_ext v0.0.0 (file:///home/user/rust/src/libsyntax_ext)
error: could not exec the linker `cc`
  |
  = note: Cannot allocate memory (os error 12)
  = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.rustc0-81414edc9fa898f39b569bc75d0a157e.rs.rcgu.o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.rustc1-81414edc9fa898f39b569bc75d0a157e.rs.rcgu.o" "-o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc-c6263d330d2b9f31.so" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.crate.metadata.rcgu.o" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc-c6263d330d2b9f31.crate.allocator.rcgu.o" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/backtrace-sys-efe11c48d393cb3a/out/.libs" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/miniz-sys-ed3f79652d61a4a4/out" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "test-902559d4a0cc83d5" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "term-36880752958d2c6f" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libflate2-be463fa6b96a16ec.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libminiz_sys-faafc4fdd43b8a1b.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbacktrace-447434a17b628693.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbacktrace_sys-63d1b058c70a5ba5.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/librustc_demangle-ffb675b42151a6f8.rlib" "-Wl,--no-whole-archive" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libbyteorder-f652c706c62d3d96.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "proc_macro-c7d45a460cb3a738" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/libjobserver-5189829dca651532.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "rustc_const_math-e212b3283c198d45" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "syntax-e97e6d4ccc983706" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_errors-dafdecc781917b18" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "syntax_pos-ccbedf94f6bd555f" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_target-aaec33fa4a23ec51" "-Wl,-Bstatic" "-Wl,--whole-archive" "/tmp/rustc.BU4zySi8u8L1/liblazy_static-d0629a75f2783580.rlib" "-Wl,--no-whole-archive" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-Wl,-Bdynamic" "-l" "graphviz-06bb7008d45cdff0" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "fmt_macros-7fa0522df5af949a" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "arena-16294da79648b864" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_data_structures-22e458fc7e0d421f" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "rustc_cratesio_shim-7bdcd669c7579602" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps" "-l" "serialize-b63c42e100577192" "-Wl,--start-group" "-L" "/home/user/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-l" "std-b63203ee1146648a" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/rustc.BU4zySi8u8L1/libcompiler_builtins-bbf524080a1524b2.rlib" "-Wl,-Bdynamic" "-l" "util" "-l" "util" "-l" "dl" "-l" "rt" "-l" "pthread" "-l" "pthread" "-l" "gcc_s" "-l" "c" "-l" "m" "-l" "rt" "-l" "pthread" "-l" "util" "-l" "util" "-shared" "-Wl,-rpath,$ORIGIN/../lib"

error: aborting due to previous error

error: Could not compile `rustc`.

Caused by:
  process didn't exit successfully: `/home/user/rust/build/bootstrap/debug/rustc --crate-name rustc librustc/lib.rs --error-format json --crate-type dylib --emit=dep-info,link -C prefer-dynamic -C opt-level=2 -C metadata=c6263d330d2b9f31 -C extra-filename=-c6263d330d2b9f31 --out-dir /home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/release/deps --extern rustc_target=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_target-aaec33fa4a23ec51.so --extern proc_macro=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libproc_macro-c7d45a460cb3a738.so --extern byteorder=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbyteorder-f652c706c62d3d96.rlib --extern lazy_static=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/liblazy_static-d0629a75f2783580.rlib --extern rustc_apfloat=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_apfloat-262722bdf7d51c3a.rlib --extern bitflags=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbitflags-fd8a318e919a8b90.rlib --extern jobserver=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libjobserver-5189829dca651532.rlib --extern graphviz=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libgraphviz-06bb7008d45cdff0.so --extern rustc_const_math=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_const_math-e212b3283c198d45.so --extern rustc_data_structures=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-22e458fc7e0d421f.so --extern fmt_macros=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libfmt_macros-7fa0522df5af949a.so --extern log=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/liblog-19a330a6ee3947d3.rlib --extern rustc_errors=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-dafdecc781917b18.so --extern syntax_pos=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax_pos-ccbedf94f6bd555f.so --extern arena=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libarena-16294da79648b864.so --extern tempdir=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libtempdir-83e93e20d067e6f5.rlib --extern syntax=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax-e97e6d4ccc983706.so --extern backtrace=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libbacktrace-447434a17b628693.rlib --extern flate2=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libflate2-be463fa6b96a16ec.rlib --extern serialize=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-b63c42e100577192.so --extern serialize=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-b63c42e100577192.rlib -L native=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/backtrace-sys-efe11c48d393cb3a/out/.libs -L native=/home/user/rust/build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/build/miniz-sys-ed3f79652d61a4a4/out` (exit code: 101)

Activity

  1. ishitatsuyuki
    added
    regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.
    I-compilememIssue: Problems and improvements with respect to memory usage during compilation.
    on May 2, 2018

  2. ishitatsuyuki commented on May 2, 2018

    @ishitatsuyuki
    ishitatsuyuki
    on May 2, 2018
    ContributorAuthor

    Although I'm not absolutely confident as I haven't performed a bisect, and perf data is not updated yet as well.

  3. nikomatsakis commented on May 2, 2018

    @nikomatsakis
    nikomatsakis
    on May 2, 2018
    Contributor

    @ishitatsuyuki

    It's back

    What exactly are you doing to reproduce this? Is that a reference to an earlier issue? (I vaguely remember an issue about this maybe...?)

  4. nikomatsakis
    added
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    on May 2, 2018

  5. ishitatsuyuki commented on May 3, 2018

    @ishitatsuyuki
    ishitatsuyuki
    on May 3, 2018
    ContributorAuthor

    @nikomatsakis It's a reference to #50073. Basically, I mean that the OOM error happened again on that environment:

    On the CircleCI Docker remote machine which has 2 cores and 8G memory, I'm no longer able to compile rustc. This error consistently occurs in stage1.

  6. nikomatsakis commented on May 3, 2018

    @nikomatsakis
    nikomatsakis
    on May 3, 2018
    Contributor

    @ishitatsuyuki thanks! perf is still not updated, it seems. #50197 could certainly be the cause, that could increase the size of the interning tables.

  7. nikomatsakis commented on May 3, 2018

    @nikomatsakis
    nikomatsakis
    on May 3, 2018
    Contributor

    triage: P-high -- we're investigating. First thing is to get the perf server up and going again, probably. ( cc @Mark-Simulacrum )

  8. nikomatsakis
    added
    P-highHigh priority
    on May 3, 2018
  9. emilyalbini
    added this to the 1.27 milestone on May 7, 2018

  10. alexcrichton commented on May 7, 2018

    @alexcrichton
    alexcrichton
    on May 7, 2018
    Member

    Scanning the current max-rss statistics of perf.rust-lang.org doesn't show much that's suspicious (no recent spikes up)

  11. ishitatsuyuki commented on May 7, 2018

    @ishitatsuyuki
    ishitatsuyuki
    on May 7, 2018
    ContributorAuthor

    What I currently know is that compiling stage1 is taking much more memory at peak, compared to stage0. It's like 5G vs 3G.

  12. ishitatsuyuki commented on May 9, 2018

    @ishitatsuyuki
    ishitatsuyuki
    on May 9, 2018
    ContributorAuthor

    As of today it seems to compile successfully; so this is no longer critical. I will reopen if this happens again.

  13. ishitatsuyuki
    closed this as completedon May 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    I-compilememIssue: Problems and improvements with respect to memory usage during compilation.P-highHigh priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.regression-from-stable-to-nightlyPerformance or correctness regression from stable to nightly.

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions