Skip to content

Commit 7a94540

Browse files
zephyriezephyrie
authored
Merge pull request #73 from Project-MONAI/update-evaluation-benchmark-wg
Update Evaluation and Benchmarking Working Group page content per fee…
2 parents 333b7f7 + 357d8cb commit 7a94540

File tree

1 file changed

+66
-49
lines changed

1 file changed

+66
-49
lines changed

‎wg_evaluation_benchmark.html‎

Lines changed: 66 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@
2121
<main class="flex-grow">
2222
<!-- Hero Section -->
2323
<section id="hero" class="py-24 bg-brand-light relative overflow-hidden">
24-
<div class="absolute inset-0 opacity-[0.10] bg-[radial-gradient(circle_at_1px_1px,#000_1px,transparent_0)] bg-[length:36px_36px]"></div>
2524
<div class="container relative">
2625
<div class="flex flex-wrap items-center">
2726
<div class="w-full max-w-3xl">
@@ -36,51 +35,67 @@ <h2 class="text-neutral-darkestblack text-2xl font-bold mb-4">Mission Statement<
3635
</div>
3736
</section>
3837

39-
<!-- Initiatives Section -->
38+
<!-- Highlights Section -->
4039
<section class="py-24 relative overflow-hidden">
4140
<div class="container relative">
42-
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Initiatives</h2>
41+
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Highlights</h2>
4342
<div class="grid grid-cols-1 md:grid-cols-2 gap-8 mt-8">
4443
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
45-
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Best Practices & Infrastructure</h3>
44+
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Recommendations</h3>
4645
<ul class="space-y-4 text-gray-700">
4746
<li class="flex items-start">
4847
<span class="text-brand-primary mr-3">•</span>
49-
<span>Development of algorithm validation standards</span>
50-
</li>
51-
<li class="flex items-start">
52-
<span class="text-brand-primary mr-3">•</span>
53-
<span>Implementation of automated evaluation systems</span>
48+
<a href="https://www.nature.com/articles/s41592-023-02151-z" class="link-hover">
49+
<span>Metrics Reloaded: recommendations for image analysis validation</span>
50+
</a>
5451
</li>
5552
<li class="flex items-start">
5653
<span class="text-brand-primary mr-3">•</span>
57-
<span>Creation of benchmark frameworks and protocols</span>
54+
<a href="https://www.nature.com/articles/s41592-023-02150-0" class="link-hover">
55+
<span>Understanding metric-related pitfalls in image analysis validation</span>
56+
</a>
5857
</li>
5958
<li class="flex items-start">
6059
<span class="text-brand-primary mr-3">•</span>
61-
<span>Establishment of quality control guidelines</span>
60+
<a href="https://metrics-reloaded.dkfz.de/" class="link-hover">
61+
<span>Metrics Reloaded Toolkit</span>
62+
</a>
6263
</li>
6364
</ul>
6465
</div>
6566

6667
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
67-
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Research & Community Standards</h3>
68+
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Implementation of recommendations</h3>
6869
<ul class="space-y-4 text-gray-700">
6970
<li class="flex items-start">
7071
<span class="text-brand-primary mr-3">•</span>
71-
<span>Development of novel evaluation metrics</span>
72+
<a href="https://github.com/Project-MONAI/MetricsReloaded/" class="link-hover">
73+
<span>MONAI Evaluation Metrics</span>
74+
</a>
7275
</li>
7376
<li class="flex items-start">
7477
<span class="text-brand-primary mr-3">•</span>
75-
<span>Implementation of performance analysis tools</span>
78+
<a href="https://docs.monai.io/en/latest/metrics.html" class="link-hover">
79+
<span>Metrics Documentation</span>
80+
</a>
7681
</li>
82+
</ul>
83+
</div>
84+
85+
<div class="md:col-span-2 bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
86+
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Related resources</h3>
87+
<ul class="space-y-4 text-gray-700">
7788
<li class="flex items-start">
7889
<span class="text-brand-primary mr-3">•</span>
79-
<span>Establishment of data sharing practices</span>
90+
<a href="https://www.dkfz.de/en/imsy/research/biomedical-image-analysis-challenges-bias-initiative" class="link-hover">
91+
<span>Biomedical Image Analysis ChallengeS (BIAS) Initiative</span>
92+
</a>
8093
</li>
8194
<li class="flex items-start">
8295
<span class="text-brand-primary mr-3">•</span>
83-
<span>Creation of reproducibility guidelines</span>
96+
<a href="https://www.rankings-reloaded.de/" class="link-hover">
97+
<span>Rankings Reloaded</span>
98+
</a>
8499
</li>
85100
</ul>
86101
</div>
@@ -90,7 +105,6 @@ <h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-ne
90105

91106
<!-- Group Leads Section -->
92107
<section class="py-24 bg-neutral-lightgray relative overflow-hidden">
93-
<div class="absolute inset-0 opacity-[0.10] bg-[radial-gradient(circle_at_1px_1px,#000_1px,transparent_0)] bg-[length:36px_36px]"></div>
94108
<div class="container relative">
95109
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Group Leads</h2>
96110
<div class="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 gap-8">
@@ -151,75 +165,78 @@ <h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-ne
151165
</div>
152166
</section>
153167

154-
<!-- Resources Section -->
168+
<!-- Ongoing Projects Section -->
155169
<section class="py-24 bg-neutral-lightgray relative overflow-hidden">
156-
<div class="absolute inset-0 opacity-[0.10] bg-[radial-gradient(circle_at_1px_1px,#000_1px,transparent_0)] bg-[length:36px_36px]"></div>
157170
<div class="container relative">
158-
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Resources</h2>
171+
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Ongoing Projects</h2>
159172
<div class="grid grid-cols-1 md:grid-cols-2 gap-8 mt-8">
160173
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
161-
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Development Resources</h3>
174+
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Reporting Guidelines Taskforce (Lead - Olivier Colliot)</h3>
162175
<ul class="space-y-4 text-gray-700">
163176
<li class="flex items-start">
164177
<span class="text-brand-primary mr-3">•</span>
165-
<a href="https://github.com/Project-MONAI/MetricsReloaded/" class="link-hover">
166-
<span>Metrics Reloaded</span>
167-
</a>
178+
<span>Surveying current reporting practices and identifying areas for improvement</span>
179+
</li>
180+
<li class="flex items-start">
181+
<span class="text-brand-primary mr-3">•</span>
182+
<span>Development of guidelines around results reporting with a focus statistical aspects</span>
183+
</li>
184+
<li class="flex items-start">
185+
<span class="text-brand-primary mr-3">•</span>
186+
<span>Identification of proper calculation and methods for various procedures (e.g., confidence intervals) across different tasks and validation metrics</span>
187+
</li>
188+
<li class="flex items-start">
189+
<span class="text-brand-primary mr-3">•</span>
190+
<span>Implementation of recommended calculations for MONAI users</span>
168191
</li>
169192
</ul>
170193
</div>
171194

172195
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
173-
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Publications</h3>
196+
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Benchmarking Datasets Taskforce (Lead - Michela Antonelli)</h3>
174197
<ul class="space-y-4 text-gray-700">
175198
<li class="flex items-start">
176199
<span class="text-brand-primary mr-3">•</span>
177-
<a href="https://www.nature.com/articles/s41592-023-02151-z" class="link-hover">
178-
<span>Metrics Reloaded: Recommendations for image-analysis validation</span>
179-
</a>
200+
<span>Data quality review for MICCAI 2025 lighthouse challenges</span>
180201
</li>
181202
<li class="flex items-start">
182203
<span class="text-brand-primary mr-3">•</span>
183-
<a href="https://www.nature.com/articles/s41592-023-02150-0" class="link-hover">
184-
<span>Metrics Reloaded: Pitfalls and recommendations for image analysis validation</span>
185-
</a>
204+
<span>Identification of key characteristics for benchmarking datasets</span>
186205
</li>
187-
</ul>
188-
</div>
189-
</div>
190-
</div>
191-
</section>
192-
193-
<!-- Collaboration Opportunities Section -->
194-
<section class="py-24 bg-white">
195-
<div class="container">
196-
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Collaboration Opportunities</h2>
197-
<div class="grid grid-cols-1 md:grid-cols-2 gap-8 mt-8">
198-
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
199-
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Development Contributions</h3>
200-
<ul class="space-y-4 text-gray-700">
201206
<li class="flex items-start">
202207
<span class="text-brand-primary mr-3">•</span>
203-
<span>Contribute to evaluation metrics development</span>
208+
<span>Encouragement to develop new datasets according to best practice</span>
204209
</li>
205210
<li class="flex items-start">
206211
<span class="text-brand-primary mr-3">•</span>
207-
<span>Help create benchmarking tools</span>
212+
<span>Identification of relevant historical datasets to be used for benchmarking</span>
208213
</li>
209214
<li class="flex items-start">
210215
<span class="text-brand-primary mr-3">•</span>
211-
<span>Develop validation frameworks</span>
216+
<span>Implementation of guidelines for upcoming datasets</span>
212217
</li>
213218
</ul>
214219
</div>
215-
220+
</div>
221+
</div>
222+
</section>
223+
224+
<!-- Collaboration Opportunities Section -->
225+
<section class="py-24 bg-white relative overflow-hidden">
226+
<div class="container relative">
227+
<h2 class="text-3xl font-bold text-neutral-darkestblack mb-8">Collaboration Opportunities</h2>
228+
<div class="grid grid-cols-1 gap-8 mt-8">
216229
<div class="bg-white p-8 rounded-lg border border-neutral-200 hover:border-brand-primary hover:shadow-md transition-all duration-300">
217230
<h3 class="text-xl font-semibold text-brand-primary mb-6 pb-2 border-b border-neutral-200">Community Engagement</h3>
218231
<ul class="space-y-4 text-gray-700">
219232
<li class="flex items-start">
220233
<span class="text-brand-primary mr-3">•</span>
221234
<span>Join our regular surveys</span>
222235
</li>
236+
<li class="flex items-start">
237+
<span class="text-brand-primary mr-3">•</span>
238+
<span>Contribute to evaluation metrics testing</span>
239+
</li>
223240
<li class="flex items-start">
224241
<span class="text-brand-primary mr-3">•</span>
225242
<span>Share your expertise in validation and benchmarking</span>

0 commit comments

Comments
 (0)