{"id":1733,"date":"2026-05-12T09:17:03","date_gmt":"2026-05-12T09:17:03","guid":{"rendered":"https:\/\/hria.utcluj.ro\/?p=1733"},"modified":"2026-05-18T20:03:20","modified_gmt":"2026-05-18T20:03:20","slug":"modele-multimodale-neuro-simbolice-image-captioning-si-kblam-in-domeniul-medical","status":"publish","type":"post","link":"https:\/\/hria.utcluj.ro\/en\/modele-multimodale-neuro-simbolice-image-captioning-si-kblam-in-domeniul-medical\/","title":{"rendered":"Modele Multimodale Neuro-Simbolice: Image Captioning \u0219i KBLaM \u00een Domeniul Medical"},"content":{"rendered":"

Medical Image Captioning: Arhitecturi ImageHRM \u0219i ImageTRM<\/strong><\/h2>\n\n\n\n

Cercetarea se concentreaz\u0103 pe dezvoltarea unor modele avansate pentru generarea de descrieri precise ale imaginilor medicale, utiliz\u00e2nd structuri ierarhice \u0219i recursive<\/sup><\/sup><\/sup><\/sup>.<\/p>\n\n\n\n

ImageHRM (Hierarchical Reasoning Model)<\/strong><\/h3>\n\n\n\n
    \n
  • Integrare:<\/strong> Conecteaz\u0103 un codificator vizual (vision encoder<\/em>) direct \u00een procesul de ra\u021bionament al modelului HRM.<\/li>\n\n\n\n
  • Module de Sistem:<\/strong>\n
      \n
    • Vision Module:<\/strong> Extrage tr\u0103s\u0103turile imaginii.<\/li>\n\n\n\n
    • L-Module:<\/strong> Realizeaz\u0103 calcule locale \u0219i ra\u021bionament.<\/li>\n\n\n\n
    • M-Module:<\/strong> Integreaz\u0103 rezultatele \u0219i actualizeaz\u0103 memoria latent\u0103 printr-o bucl\u0103 repetitiv\u0103.<\/li>\n\n\n\n
    • H-Module:<\/strong> Genereaz\u0103 decizia sau predic\u021bia de nivel \u00eenalt c\u0103tre capul de ie\u0219ire (Head<\/em>).<\/li>\n<\/ul>\n<\/li>\n\n\n\n
    • Clustering Semantic:<\/strong> Include o bucl\u0103 intermediar\u0103 pentru a face tranzi\u021bia \u00eentre reprezent\u0103rile de nivel \u00eenalt (HL) \u0219i cele medii-\u00eenalte (HML).<\/li>\n\n\n\n
    • Backbones Testate:<\/strong> ResNet18, Swin Transformer \u0219i FuseLIP. Modulele vizuale pre-antrenate sunt men\u021binute \u201e\u00eenghe\u021bate\u201d (frozen<\/em>) pentru a p\u0103stra cuno\u0219tin\u021bele vizuale, \u00een timp ce nucleul de ra\u021bionament \u00eenva\u021b\u0103 structurarea textului clinic.<\/li>\n<\/ul>\n\n\n
      \n
      \"\"<\/figure>\n<\/div>\n\n\n

      ImageTRM (Tiny Recursive Model)<\/strong><\/h3>\n\n\n\n
        \n
      • Arhitectur\u0103:<\/strong> Un Transformer cu 2 straturi care ruleaz\u0103 recursiv de $N$ ori cu greut\u0103\u021bi partajate.<\/li>\n\n\n\n
      • Eficien\u021b\u0103:<\/strong> Amprent\u0103 redus\u0103 a modelului, av\u00e2nd doar aproximativ 7 milioane de parametri.<\/li>\n\n\n\n
      • Stabilitate:<\/strong> Utilizeaz\u0103 netezirea greut\u0103\u021bilor de tip EMA (Exponential Moving Average<\/em>) pentru a men\u021bine stabilitatea recursiunii ad\u00e2nci \u00een timpul antren\u0103rii.<\/li>\n\n\n\n
      • Performan\u021b\u0103 SOTA:<\/strong> Ob\u021bine cele mai bune rezultate comparativ cu variantele ImageHRM, ating\u00e2nd un scor CIDEr de 0.449<\/strong> \u0219i ROUGE-L de 0.199<\/strong> folosind un backbone Swin.<\/li>\n<\/ul>\n\n\n
        \n
        \"\"<\/figure>\n<\/div>\n\n\n
        \n\n\n\n

        Knowledge Base Augmented Language Models (KBLaM) Transferabile<\/strong><\/h2>\n\n\n\n

        Este explorat\u0103 portabilitatea modelelor de limbaj augmentate cu baze de cuno\u0219tin\u021be \u00eentre diferite domenii medicale \u0219i ontologii.<\/p>\n\n\n\n

        Metodologie \u0219i Observa\u021bii<\/strong><\/h3>\n\n\n\n
          \n
        • Antrenare:<\/strong> Modelul KBLaM este antrenat exclusiv pe setul de date DOID<\/strong> (BioML).<\/li>\n\n\n\n
        • Inferen\u021b\u0103 Cross-Domain:<\/strong> Modelul este aplicat f\u0103r\u0103 reantrenare pe alte ontologii \u0219i arii medicale, precum NCIT<\/strong>, ORDO<\/strong> sau OMIM<\/strong>.<\/li>\n\n\n\n
        • Transferabilitate:<\/strong> S-a observat c\u0103 mecanismul de aten\u021bie antrenat pe o ontologie func\u021bioneaz\u0103 eficient pe ontologii diferite, ceea ce demonstreaz\u0103 c\u0103 modelul \u00eenva\u021b\u0103 structura aten\u021biei independent de setul de date.<\/li>\n<\/ul>\n\n\n\n

          Rezultate Inferen\u021b\u0103 (Accurate\u021be)<\/strong><\/h3>\n\n\n\n

          Performan\u021ba a fost evaluat\u0103 pe diverse ontologii cu dimensiuni diferite ale bazei de cuno\u0219tin\u021be (KB size):<\/p>\n\n\n

          \n
          \"\"<\/figure>\n<\/div>\n\n\n

          <\/p>","protected":false},"excerpt":{"rendered":"

          Medical Image Captioning: Arhitecturi ImageHRM \u0219i ImageTRM Cercetarea se concentreaz\u0103 pe dezvoltarea unor modele avansate pentru generarea de descrieri precise ale imaginilor medicale, utiliz\u00e2nd structuri ierarhice \u0219i recursive. ImageHRM (Hierarchical Reasoning Model) ImageTRM (Tiny Recursive Model) Knowledge Base Augmented Language Models (KBLaM) Transferabile Este explorat\u0103 portabilitatea modelelor de limbaj augmentate cu baze de cuno\u0219tin\u021be \u00eentre… <\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"default","neve_meta_enable_content_width":"on","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"off","neve_post_elements_order":"[\"content\"]","neve_meta_disable_header":"off","neve_meta_disable_footer":"","neve_meta_disable_title":"","_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-1733","post","type-post","status-publish","format-standard","hentry","category-rezultate"],"_links":{"self":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1733","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/comments?post=1733"}],"version-history":[{"count":6,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1733\/revisions"}],"predecessor-version":[{"id":1843,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1733\/revisions\/1843"}],"wp:attachment":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/media?parent=1733"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/categories?post=1733"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/tags?post=1733"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}