Deep Dive into Codatta’s Medical Image Annotation Platform: A Proposal with Contributors’ Best Interests in Mind
TL;DR:
Codatta’s medical image annotation platform uses blockchain to enhance the quality and distrbution of pathology image annotations. It features a hierarchical management system with roles such as General Manager and Organ Managers, and a contributor base consisting of medical students and pathologists, ranked by skill level and reputation. The platform’s tokenomics model rewards contributors based on their skill level, time spent, and the accuracy of their work. It includes a dynamic validation process that adjusts rewards based on community consensus. Codatta promotes transparency and societal impact, and is developing educational tools and a pathology image search engine to further contribute to medical education and patient care. Additionally, it provides statistics about how annotated data is utilized in academic research, education, and AI model development, ensuring that contributors feel a direct impact on the advancement of healthcare.
Introduction
In the rapidly evolving landscape of medical AI, high-quality annotated pathology images are becoming increasingly valuable. However, traditional annotation platforms often struggle with issues of data quality, fair compensation, and long-term contributor engagement. Codatta, a revolutionary blockchain-based platform, is designed to transform the way pathology image annotations are aggregated, validated, and monetized. As it is shown in Figure 1, pathologists can readily earn income in both short term and long term by contributing annotations and reviewing others’ annotations. Their long-term earnings are safeguarded through the establishment of data ownership rights by staking within the platform, which becomes a valuable asset for SaaS companies that utilize these databases and for AI developers that use the data for model training. This revenue model ensures that contributors are rewarded for the enduring value their contributions provide to society.
Platform Structure
Codatta ecosystem is built on a hierarchical structure that ensures efficient management and quality control:
- General Manager (GM): Oversees the entire platform operation.
- Organ Managers (OMs): Manage specific datasets under biological organs, making tutorials and determining skill requirements for annotation tasks.
- Contributors: Primarily medical students and pathologists who annotate, validate, or invalidate submissions.
Contributors are classified into five skill levels: junior, experienced, senior, expert, and grandmaster. This classification ensures those with appropriate expertise handle complex tasks.
The Annotation Process
- Managers issue annotation tasks with requirements and tutorials.
- Contributors, who are randomly picked, view available tasks matching their specialty and skill level.
- To accept a task, contributors must stake CODAs, demonstrating their confidence and commitment, see Figure 2.
- Contributors may reject a task due to insufficient expertise or uncertainty regarding the data’s worth.
5. After annotation, the submission goes through a multi-level validation process.
6. Validated annotations are added to the dataset, and contributors receive CODA rewards.
7. CODA received by contributors can be used to stake on the annotation validated to obtain ownership, as shown in Figure 3.
8. By investing in a variety of annotated datasets through staking, a contributor can curate a diverse portfolio of annotation assets. These assets have the potential to yield a steady income stream and offer the flexibility to be traded in the marketplace, providing contributors with both liquidity and an ongoing return on their investment.
Proposed CODA Tokenomics Model
Here we illustrate two of the most important aspects of our tokenomics model.
- Initial Contribution Valuation & Reward Allocation
Calculate the total value of contributions by determining the average of skill-level multiplied by time spent for all contributors. Specifically, we can define the following variables:
This formula promotes fairness by ensuring that contributors who are more skilled and spend more time on the task receive a larger share of the reward pool. It also encourages quality work on complicated cases, as the time spent should correlate with the difficulty and complexity of the annotations. However, the rewards are not allocated until validation.
2. Majority Voting and Validation
The majority vote process is a critical component in the fractional ownership determination, especially in cases where annotations may be disputed or require consensus validation. The goal of this process is to adjust the ownership fractions based on community consensus, ensuring that the final ownership reflects the collective judgment of the contributors. Figure 4 below demonstrates the details for the majority voting process.
In Codatta, a contributor stakes some tokens when accepting an annotation task. If their annotations are found to be significantly incorrect based on the community vote or expert review, a portion of these tokens will be burnt (removed from circulation) as a penalty.
The majority vote will determine a winner, i.e., the most accurate or reliable set annotation from a particular contributor. The platform will then compare each contributor’s annotations with the winner’s to adjust reward weights. The weight adjustment will prioritize the following aspects, in order:
a. Labeling Consistency: If a contributor’s annotations match the winner’s label. Note that labels can be begin/malignant, tumor subtype, metastatic, or not, depending on the task requirements.
b. Regional Delineation Overlap: Measured by the Dice score, which quantifies the overlap between delineated regions of different contributors.
c. Description Accuracy: The accuracy of textual diagnostic description with regard to the winner’s description can be measured using medical LLM.
As it is depicted in Figure 4, three contributors in the mahority voting process get the diagnostic label right while one gets slashed. The consensus is reached when a senior contributor validates the winner’s answer. Subsequently, the rewards are determined, to reflect the collective assessment made by the contributors.
Weight Adjustment Formula:
Recalculation of Reward Fractions:
After the weights are adjusted, recalculate the reward fractions to reflect the new weights:
This ensures that the reward fractions are updated to align with the community’s consensus and the quality of each contributor’s work. By implementing these improvements, the Codatta platform will encourage accuracy and consistency in annotations, penalize poor-quality contributions, and reward annotators who provide high-quality, consistent, and reliable work. This recalculation of reward fractions will further enhance the trust and reliability of the dataset and the fairness of the reward system.
Societal Impact and Educational Benefits
Codatta’s community of pathologists and medical students can also find profound societal rewards in their contributions. Guided by the commitment to heal and help, their work on the platform transcends financial incentives. Enhancing AI diagnostics and medical research directly influences patient outcomes and contributes to the betterment of healthcare.
The Codatta platform ensures that contributors are aware of the impact of their work through transparency on data utilization (see Figure 5). It provides a comprehensive view of how their annotations are applied in diagnostic AI models and medical research. Note that informativeness is measured by the number of cases in the dataset with complete annotation, including diagnostic label, regional delineation, and textual description. Confidence is measured by the number of submitted annotations that are validated by senior contributors with high reputation and skill-levels.
Moreover, Codatta platform will develop two exemplary applications: an Educational Academy, which will enhance the learning experience for medical students with real-world and fully annotated pathology data, and an AI-based Pathology Image Search Engine, designed to assist clinicians in pattern recognition and diagnosis. This tool is particularly beneficial for junior pathologists, providing them with critical support in their diagnostic processes.
Therefore, contributors are not just part of a dataset; they are at the heart of a clinical ecosystem that feeds back into their professional environment, creating a cycle of continuous learning and improvement. This cycle empowers the community and ensures that their contributions are part of a lasting legacy in medical advancement.
Conclusion
The Codatta platform represents a paradigm shift in pathology image annotation. By leveraging blockchain technology, tokenized incentives, and a novel ownership model, it creates a self-sustaining ecosystem that addresses the key challenges of data quality, fair compensation, and long-term engagement. Also, Codatta offers medical professionals a platform that aligns with their ethical commitments, provides intrinsic fulfillment through societal impact, and directly contributes to the advancement of medical education and clinical practice. In essence, Codatta is poised to play a crucial role in advancing medical AI and improving patient care worldwide.
About codatta
Codatta is a universal annotation and labeling platform that turns your intelligence into AI.
Our mission is to lower the barrier for AI development teams by providing inclusive access to quality data, facilitating AI advancement, and to empower individuals to contribute to AI development and enjoy long-lasting rewards for their critical contributions. We tackle challenges across various verticals, including crypto (account and user annotation), healthcare, and robotics. Our user-contributed data is on the right track to commercialization in areas like web3 ads, AML, and healthcare.
Stay Connected with codatta
Follow us on social media for the latest news, insights, and developments about our innovative projects. Join our growing community below and don’t forget to like, comment, and share our posts to help spread the word!
🌐 Website｜🆇 Twitter｜💬 Telegram｜👾 Discord｜📱App
