Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove nsys report from gpu_aquaplanet_dyamond #3358

Merged
merged 1 commit into from
Oct 3, 2024

Conversation

charleskawczynski
Copy link
Member

This PR removes the nsight from from gpu_aquaplanet_dyamond, since it's giving some errors while saving profiler information and then appears to be hanging when uploading to buildkite (example here):

Saving profiler information | 1m 42s
-- | --
  | Generating '/tmp/slurm-44215412/nsys-report-9f55.qdstrm'
  | [1/1] [==19%                       ] report.nsys-rep
  | Importer error status: Importation failed.
  | Import Failed with unexpected exception: /dvs/p4/build/sw/devtools/Agora/Rel/QuadD_Main/QuadD/Host/QdstrmImporter/main.cpp(34): Throw in function {anonymous}::Importer::Importer(const boost::filesystem::path&, const boost::filesystem::path&)
  | Dynamic exception type: boost::wrapexcept<QuadDCommon::RuntimeException>
  | std::exception::what: RuntimeException
  | [QuadDCommon::tag_message*] = Status: AnalysisFailed
  | Error {
  | Type: RuntimeError
  | SubError {
  | Type: InvalidArgument
  | Props {
  | Items {
  | Type: OriginalExceptionClass
  | Value: "N5boost10wrapexceptIN11QuadDCommon24InvalidArgumentExceptionEEE"
  | }
  | Items {
  | Type: OriginalFile
  | Value: "/dvs/p4/build/sw/devtools/Agora/Rel/QuadD_Main/QuadD/Host/Analysis/Modules/EventCollection.cpp"
  | }
  | Items {
  | Type: OriginalLine
  | Value: "1055"
  | }
  | Items {
  | Type: OriginalFunction
  | Value: "void QuadDAnalysis::EventCollection::CheckOrder(QuadDAnalysis::EventCollectionHelper::EventContainer&, const QuadDAnalysis::ConstEvent&) const"
  | }
  | Items {
  | Type: ErrorText
  | Value: "Wrong event order has been detected when adding events to the collection:\nnew event ={ StartNs=282462349596 StopNs=282462349705 GlobalId=297315521685546 Event={ TraceProcessEvent=[{ Correlation=18871 EventClass=1 TextId=706 ReturnValue=0 },] } Type=48 }\nlast event ={ StartNs=724295437177 StopNs=724296018404 GlobalId=297315521685546 Event={ TraceProcessEvent=[{ Correlation=96667 EventClass=1 TextId=708 ReturnValue=0 },] } Type=48 }"
  | }
  | }
  | }
  | }
  | Generated:
  | /central/scratch/esm/slurm-buildkite/climaatmos-ci/20961/climaatmos-ci/gpu_aquaplanet_dyamond/output_active/report.qdstrm
  | Running global post-command hook | 0s
  | Uploading artifacts |  
  | $ buildkite-agent artifact upload gpu_aquaplanet_dyamond/output_active/*
  | 2024-10-03 07:53:56 INFO   Found 8 files that match "gpu_aquaplanet_dyamond/output_active/*"
  | 2024-10-03 07:53:56 INFO   Uploading to default Buildkite artifact storage
  | 2024-10-03 07:53:56 INFO   Creating (0-8)/8 artifacts
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-923a-421a-a27d-c37b0ed36f28 gpu_aquaplanet_dyamond/output_active/summary.pdf (364 KiB)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9234-4068-a43e-e688fa6bb057 gpu_aquaplanet_dyamond/output_active/gpu_aquaplanet_dyamond_parameters.toml (25 KiB)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9234-4859-a4e7-ea32c6813dad gpu_aquaplanet_dyamond/output_active/hdf5_files.tar (512 B)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9235-44a1-a9fa-56f05c28b886 gpu_aquaplanet_dyamond/output_active/summary_2D.pdf (1.6 MiB)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9237-487b-9e0a-89f11c2430ab gpu_aquaplanet_dyamond/output_active/report.qdstrm (222 MiB)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9238-4352-abc8-a773bb5065b1 gpu_aquaplanet_dyamond/output_active/graceful_exit.dat (1 B)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9238-43de-966c-916f33924a79 gpu_aquaplanet_dyamond/output_active/nc_files.tar (378 MiB)
  | 2024-10-03 07:53:56 INFO   Uploading artifact 019252df-9239-4312-8d85-495285bc93fc gpu_aquaplanet_dyamond/output_active/gpu_aquaplanet_dyamond.yml (2.7 KiB)
  | 2024-10-03 07:53:56 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/gpu_aquaplanet_dyamond.yml"
  | 2024-10-03 07:53:56 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/hdf5_files.tar"
  | 2024-10-03 07:53:57 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/graceful_exit.dat"
  | 2024-10-03 07:53:57 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/gpu_aquaplanet_dyamond_parameters.toml"
  | 2024-10-03 07:53:57 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/summary.pdf"
  | 2024-10-03 07:54:01 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/summary_2D.pdf"
  | 2024-10-03 08:17:35 INFO   Successfully uploaded artifact "gpu_aquaplanet_dyamond/output_active/report.qdstrm"

@charleskawczynski charleskawczynski merged commit b17acc1 into main Oct 3, 2024
13 of 14 checks passed
@charleskawczynski charleskawczynski deleted the ck/rm_nsight_report branch October 3, 2024 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants