Pediatric Sepsis Data Challenge

Once your team has registered, please head over to our challenge Github repository to access the synthetic dataset, data dictionary, and instructions. Please note that we are still in the process of finalizing the baseline model and evaluation metrics. More details on these will be provided in the next couple of weeks, so stay tuned!

Please request access to join the private Google group. Feel free to post any questions in the group, and one of the challenge organizers will assist you.

Due to the limitations we face and the environments where our models will be executed, we are currently restricting the tools to Python and R, along with packages that are widely available in these two environments. While some participants use proprietary tools like MATLAB, we are emphasizing the open-source aspect of this competition. To maintain this focus throughout the challenge, we are requiring the use of open-source programming languages and libraries.

We are placing restrictions on the types of model’s participants can use and the formal instruction will be published soon. We invite teams to explore a wide range of models, including large language models (LLMs) and other current approaches. However, models will be required to be open source and we will not be able to send the confidential data to an external source. We are particularly interested in teams that not only use advanced tools but also optimize them for practical use cases—especially in rural or resource-limited environments. In addition to maximizing algorithm performance, participants should consider the resources and strengths available in these settings.

While we are not imposing strict requirements, competitors who achieve higher accuracy with shorter run times and lower memory usage will have a competitive edge. More details will follow.

Once the preliminary phase is complete and the active phase begins, participants will have opportunities to update their submissions. While there is a limit on the number of submissions allowed, we will ask teams to select one submission to be run on the final holdout dataset. Each submission will be used to generate the official leaderboard.

We will provide more information during the preliminary phase. 

The goal of the challenge is to outperform the baseline model while balancing efficiency and computational performance.

For this challenge, we are focusing only on in-hospital mortality. The original project aimed to examine post-hospital mortality, but this dataset is limited to in-hospital mortality. Only children admitted with suspected or confirmed infections were included in this dataset. We use a broad definition of sepsis to encompass all children showing clinical signs and symptoms of infection, which is why we specifically refer to it as sepsis. Clinically, we would classify all these children as having sepsis. Since all the children were admitted to a facility, we believe this is a reasonable classification, despite the limitations of our diagnostic tools. It's important to note that children with trauma, burns, follow-up visits, and similar conditions were not included in this cohort.

We are working on it! We would welcome any feedback as we move forward with this.

There are some legitimate concerns on processing using synthetic data, but we need to keep in mind the sensitivity of the dataset including the risk of reidentification of families and children that are involved in providing this data. We are carefully assessing this risk, and throughout the competition, as we implement these synthetic data-derived models on the original dataset, we will closely monitor for any potential bias and work to correct them as best we can.

The synthetic data is generated from the real data, so we expect the structure to remain the same, though the distributions may differ slightly. We will provide a data dictionary with detailed information about the data elements, along with rankings that reflect the weighting we believe should be applied based on the difficulty of collecting each data element.

Some aspects of this have been published in our original paper, but we are being cautious about what we disclose. We want to avoid having challenge participants replicate the nuances of synthetic data generation and instead focusing on the task at hand. Throughout the challenge, we will provide feedback on any data drifts or inconsistencies we observe when replicating results on the original dataset. The primary goal of the preliminary phase is to minimize these issues and ensure submissions are aligned with the original data. Participants will be kept informed throughout this process. We should emphasize that 30% of the data is missing, not due to poor data collection, but because certain questions were only asked under specific conditions. For example, if a previous answer was negative, some follow-up questions were skipped, contributing to the missing data. We've replicated this pattern as closely as possible in the synthetic dataset.

We are not setting specific requirements for managing data imbalances. There are various ways to address class imbalances, each with its own advantages and disadvantages. We encourage teams to explore different strategies and present their best approach.

We are seeking entries or models that not only outperform competitors but also meet the other criteria. The primary focus is ensuring that the model effectively completes the main task. If multiple models achieve similar performance in predicting outcomes, we'll consider two secondary factors: computational complexity (resources required) and time complexity (how long the model takes to run). If a model takes longer but delivers superior results, it won't be penalized compared to faster models with worse performance. This is a multifactorial assessment, and once the utility function is released, you'll see how these criteria were weighted. You will also receive the scores for each of the 3 criteria (performance, computational complexity computational, time complexity). 

This goes back to the previous question around developing algorithm that are both accurate but also efficient. There may be a tradeoff, but the challenge participants will have to optimize the solution. We will be providing a more detailed benchmarking process and utility function.

More details will be shared in the coming weeks as we finalize the metrics and weighting applied within a broader utility function.

We are using a multifaceted approach for model evaluation. While this will be one factor, it will be weighted alongside other criteria.

There are a number of different opportunities for presentation and publication that need to be finalized. Stay tuned for the developments in this space. 

This is not a part of the challenge at this stage.

The primary sponsor of the Pediatric Sepsis Data CoLaboratory, from which this challenge and data originated, is the World Federation of Pediatric Intensive & Critical Care Societies (WFPICCS). We are in the early stages of planning a session at the 2026 WFPICCS World Congress. We aim to have the top-performing teams present abstracts or publish proceedings, whether at WFPICCS or another conference. Stay tuned!