{"id":1778,"date":"2026-03-09T16:22:02","date_gmt":"2026-03-09T16:22:02","guid":{"rendered":"https:\/\/hria.utcluj.ro\/?p=1778"},"modified":"2026-06-15T13:45:37","modified_gmt":"2026-06-15T13:45:37","slug":"perceptia-si-intelegerea-scenelor-3d-din-imagini-aeriene","status":"publish","type":"post","link":"https:\/\/hria.utcluj.ro\/en\/perceptia-si-intelegerea-scenelor-3d-din-imagini-aeriene\/","title":{"rendered":"Perception and understanding of 3D scenes from aerial images"},"content":{"rendered":"<p class=\"wp-block-paragraph\" id=\"p-rc_8aa72877c1b59b65-401\">The conducted research aims to create a unified framework for interpreting visual data captured by aerial platforms (UAVs), moving from pixel-level processing to complex semantic and structural representations.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Three-level perception architecture<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"p-rc_8aa72877c1b59b65-402\">The system is hierarchically organized to transform raw data into informed decisions<sup><\/sup>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"translation-block\"><strong>Level 1 \u2013 Visual Perception:<\/strong> Focuses on extracting fundamental features from RGB images or video streams. It includes dense depth predictors, panoptic segmentation methods (generating semantic and instance masks), and algorithms for object detection in 3D space.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Level 2 \u2013 Neural Scene Reconstruction:<\/strong> Uses static and dynamic neural predictors to generate 3D models. This stage enables the rendering of projective images, depth maps, orthographic views, and Bird\u2019s Eye View (BEV) projections. It also performs localization on Open Street Map (OSM)-based maps.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Level 3 \u2013 Scene Graph:<\/strong> Represents the highest level of abstraction where change detection, road property analysis, and land use detection are performed. It serves as an interface for the decision-making agent.<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"443\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture-1024x443.png\" alt=\"\" class=\"wp-image-1779\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture-1024x443.png 1024w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture-300x130.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture-768x333.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture-18x8.png 18w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/perception-architecture.png 1337w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\"><strong>The ClaraVid synthetic dataset<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph translation-block\" id=\"p-rc_8aa72877c1b59b65-406\">To train models capable of handling complex scenarios, the <strong>ClaraVid<\/strong> dataset was developed, offering major advantages over traditional datasets<sup><\/sup>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"translation-block\"><strong>Multi-View Acquisition:<\/strong> Enables simultaneous capture from multiple viewpoints (High, Mid, Low angle) for high-resolution scenarios.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Comprehensive Multimodal Annotation:<\/strong> Includes scene-level point clouds (PCL), masks for dynamic objects, panoptic segments (semantic and instance), and accurate depth maps.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Accessibility:<\/strong> The data is publicly available on the Hugging Face platform: <a href=\"http:\/\/huggingface.co\/datasets\/radubeche\/claravid\" data-type=\"link\" data-id=\"huggingface.co\/datasets\/radubeche\/claravid\" target=\"_self\">huggingface.co\/datasets\/radubeche\/claravid<\/a>.<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"1024\" height=\"495\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1-1024x495.png\" alt=\"\" class=\"wp-image-1780\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1-1024x495.png 1024w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1-300x145.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1-768x371.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1-18x9.png 18w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid1.png 1270w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img decoding=\"async\" width=\"1024\" height=\"351\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2-1024x351.png\" alt=\"\" class=\"wp-image-1781\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2-1024x351.png 1024w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2-300x103.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2-768x263.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2-18x6.png 18w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/claravid2.png 1470w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\">UAVid++ \u2013 Advanced semantic segmentation for UAV imagery<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Visual perception in aerial (UAV) scenarios poses significant challenges compared to ground-level imagery. Among the main difficulties are:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>extreme scale variations (from large buildings to very small objects such as pedestrians or distant vehicles)<\/li>\n\n\n\n<li>high density in urban areas<\/li>\n\n\n\n<li>visual ambiguities caused by observation angles and altitude<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Existing datasets, including UAVid, have important limitations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>coarse annotations (approximate labels)<\/li>\n\n\n\n<li>semantic inconsistencies between classes<\/li>\n\n\n\n<li>lack of fine-grained contour details<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These limitations directly affect the performance of segmentation models, especially regarding small objects and generalization in new scenarios.<\/p>\n\n\n\n<p class=\"wp-block-paragraph translation-block\">To overcome these limitations, we developed <strong>UAVid++<\/strong>, a significantly improved version of the UAVid dataset, which introduces:<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">High-fidelity annotations<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>manual and automatic correction of existing labels<\/li>\n\n\n\n<li>strict semantic alignment between frames<\/li>\n\n\n\n<li>much more precise pixel-level contours<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Extended semantic taxonomy<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>introduction of additional classes relevant to aerial scenarios (e.g., water, sky, rooftops)<\/li>\n\n\n\n<li>more coherent structure compatible with other UAV datasets<\/li>\n\n\n\n<li>support for out-of-distribution evaluations<\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Fine segmentation in complex scenarios<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li>improving the representation of small and crowded objects<\/li>\n\n\n\n<li>clearer separation between nearby instances<\/li>\n\n\n\n<li>reducing ambiguities in dense urban areas<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"739\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1-1024x739.png\" alt=\"\" class=\"wp-image-1782\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1-1024x739.png 1024w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1-300x216.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1-768x554.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1-18x12.png 18w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid1.png 1163w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\">Integration of Large Vision Models (LVM)<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">To fully leverage the new dataset, a hybrid architecture has been proposed that combines:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"translation-block\"><strong>DINO backbone (self-supervised Vision Transformer)<\/strong><br>\u2192 provides robust semantic representations and excellent generalization capability<\/li>\n\n\n\n<li class=\"translation-block\"><strong>U-Net-inspired head<\/strong><br>\u2192 recovers fine spatial details and improves contour precision<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">This combination addresses a fundamental trade-off in visual perception:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>large models (LVMs) \u2192 capture global context well but lose fine-grained details<\/li>\n\n\n\n<li>specialized models \u2192 provide local precision but generalize less effectively<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph translation-block\">By integrating them, we obtain a model capable of simultaneously handling <strong>global context and local detail<\/strong>.<\/p>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"854\" style=\"aspect-ratio: 2424 \/ 854;\" width=\"2424\" controls src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/small_object_segmentation.mp4\"><\/video><\/figure>\n\n\n\n<div style=\"height:31px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h4 class=\"wp-block-heading\">The conducted experiments demonstrate consistent improvements:<\/h4>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"translation-block\"><strong>Superior generalization<\/strong><br>Models trained on UAVid++ perform robustly across different scenes, altitudes, and camera orientations.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Improved small-object segmentation<\/strong><br>Detection and delineation of small-sized objects is significantly more accurate.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Clearer and more stable contours<\/strong><br>Object boundaries are better defined, reducing classification errors at borders.<\/li>\n\n\n\n<li><strong>Calitate apropiat\u0103 de ground truth<\/strong><br>Nivelul ridicat al adnot\u0103rilor permite utilizarea dataset-ului pentru:\n<ul class=\"wp-block-list\">\n<li>knowledge distillation<\/li>\n\n\n\n<li>evalu\u0103ri riguroase ale modelelor<\/li>\n\n\n\n<li>benchmark-uri realiste<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-video\"><video height=\"714\" style=\"aspect-ratio: 1632 \/ 714;\" width=\"1632\" controls src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/small_object_segmentation_results_convert.mp4\"><\/video><\/figure>\n\n\n\n<div style=\"height:32px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Monocular depth estimation and geo-informed scaling<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\" id=\"p-rc_8aa72877c1b59b65-410\">A major challenge in aerial perception is obtaining accurate metric depth from monocular images. The proposed solution includes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"translation-block\"><strong>UAVid-3D-Scenes:<\/strong> An extension of the UAVid semantic dataset focused on depth, enabling joint research on semantic and 3D tasks.<\/li>\n\n\n\n<li class=\"translation-block\"><strong>Integration of TanDEM-X GDEM:<\/strong> Global elevation data (Digital Elevation Model) is used to ensure correct metric scaling of relative depth models.<\/li>\n\n\n\n<li><strong>Fluxul de Procesare a Datelor Real-World:<\/strong>\n<ul class=\"wp-block-list\">\n<li>Alinierea coordonatelor GPS cu imagini satelitare pentru setul UAVid.<\/li>\n\n\n\n<li>Utilizarea algoritmului <strong>Cloth Simulation Filter (CSF)<\/strong> pentru segmentarea eficient\u0103 a solului.<\/li>\n\n\n\n<li>Proiec\u021bia \u0219i mascarea datelor pentru corelarea precis\u0103 \u00eentre cadrele camerei \u0219i modelul digital de teren.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"220\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-1024x220.png\" alt=\"\" class=\"wp-image-1785\" style=\"width:1170px;height:auto\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-1024x220.png 1024w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-300x64.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-768x165.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-1536x330.png 1536w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x-18x4.png 18w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/tandem-x.png 1780w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n<\/div>\n\n\n<h3 class=\"wp-block-heading\">UAVid-3D-Scenes \u2013 Depth-oriented extension<\/h3>\n\n\n\n<p class=\"wp-block-paragraph translation-block\"><strong>UAVid-3D-Scenes<\/strong> is an extension of the UAVid dataset, focused on integrating depth information to enable applications that combine semantic and 3D data. The dataset is publicly available on HuggingFace: <a href=\"https:\/\/huggingface.co\/datasets\/hrflr\/uavid-3d-scenes\" target=\"_self\">huggingface.co\/datasets\/hrflr\/uavid-3d-scenes<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The dataset includes:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>dense and sparse depth<\/li>\n\n\n\n<li>alignment of UAVid frames to global coordinates (using GPS and satellite imagery)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph translation-block\">For each frame, data from the <strong>TanDEM-X (GDEM)<\/strong> elevation model are integrated, used for terrain information projection.<\/p>\n\n\n\n<p class=\"wp-block-paragraph translation-block\">Additionally, 3D reconstructions from the <strong>UFO Depth dataset<\/strong> are used for out-of-distribution evaluations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The goal is to improve the metric scaling of depth estimates by combining position, altitude, and digital terrain model information.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"783\" height=\"280\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/ufod.png\" alt=\"\" class=\"wp-image-1786\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/ufod.png 783w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/ufod-300x107.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/ufod-768x275.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/ufod-18x6.png 18w\" sizes=\"(max-width: 783px) 100vw, 783px\" \/><\/figure>\n<\/div>\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"903\" height=\"284\" src=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid-tandemx.png\" alt=\"\" class=\"wp-image-1787\" srcset=\"https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid-tandemx.png 903w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid-tandemx-300x94.png 300w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid-tandemx-768x242.png 768w, https:\/\/hria.utcluj.ro\/wp-content\/uploads\/2026\/05\/uavid-tandemx-18x6.png 18w\" sizes=\"(max-width: 903px) 100vw, 903px\" \/><\/figure>\n<\/div>","protected":false},"excerpt":{"rendered":"<p>The conducted research aims to create a unified framework for interpreting visual data captured by UAV platforms, moving from pixel-level processing to complex semantic and structural representations. Three-level perception architecture The system is hierarchically organized to transform raw data into informed decisions: Synthetic dataset ClaraVid To train models capable of handling\u2026&nbsp;<\/p>","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"neve_meta_sidebar":"","neve_meta_container":"default","neve_meta_enable_content_width":"on","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"off","neve_post_elements_order":"[\"content\"]","neve_meta_disable_header":"off","neve_meta_disable_footer":"","neve_meta_disable_title":"","_themeisle_gutenberg_block_has_review":false,"footnotes":""},"categories":[7],"tags":[],"class_list":["post-1778","post","type-post","status-publish","format-standard","hentry","category-rezultate"],"_links":{"self":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1778","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/comments?post=1778"}],"version-history":[{"count":7,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1778\/revisions"}],"predecessor-version":[{"id":1892,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/posts\/1778\/revisions\/1892"}],"wp:attachment":[{"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/media?parent=1778"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/categories?post=1778"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hria.utcluj.ro\/en\/wp-json\/wp\/v2\/tags?post=1778"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}