The Latest Papers about AI
-
The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs
2022-08-18 17:59:41Chris Rockwell, Justin Johnson, David F. FouheyAbstract
We present a simple baseline for directly estimating the relative pose (rotation and translation, including scale) between two images. Deep methods have recently shown strong progress but often require complex or multi-stage architectures. We show that a handful of modifications can be applied to a Vision Transformer (ViT) to bring its computations close to the Eight-Point Algorithm. This inductive bias enables a simple method to be competitive in multiple settings, often substantially improving over the state of the art with strong performance gains in limited data regimes.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08988
PDF
-
Open-Vocabulary Panoptic Segmentation with MaskCLIP
2022-08-18 17:55:37Zheng Ding, Jieke Wang, Zhuowen TuAbstract
In this paper, we tackle a new computer vision task, open-vocabulary panoptic segmentation, that aims to perform panoptic segmentation (background semantic labeling + foreground instance segmentation) for arbitrary categories of text-based descriptions. We first build a baseline method without finetuning nor distillation to utilize the knowledge in the existing CLIP model. We then develop a new method, MaskCLIP, that is a Transformer-based approach using mask queries with the ViT-based CLIP backbone to perform semantic segmentation and object instance segmentation. Here we design a Relative Mask Attention (RMA) module to account for segmentations as additional tokens to the ViT CLIP model. MaskCLIP learns to efficiently and effectively utilize pre-trained dense/local CLIP features by avoiding the time-consuming operation to crop image patches and compute feature from an external CLIP image model. We obtain encouraging results for open-vocabulary panoptic segmentation and state-of-the-art results for open-vocabulary semantic segmentation on ADE20K and PASCAL datasets. We show qualitative illustration for MaskCLIP with custom categories.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08984
PDF
-
'Melatonin': A Case Study on AI-induced Musical Style
Abstract
Although the use of AI tools in music composition and production is steadily increasing, as witnessed by the newly founded AI song contest, analysis of music produced using these tools is still relatively uncommon as a mean to gain insight in the ways AI tools impact music production. In this paper we present a case study of "Melatonin", a song produced by extensive use of BassNet, an AI tool originally designed to generate bass lines. Through analysis of the artists' work flow and song project, we identify style characteristics of the song in relation to the affordances of the tool, highlighting manifestations of style in terms of both idiom and sound.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08968
PDF
-
GSRFormer: Grounded Situation Recognition Transformer with Alternate Semantic Attention Refinement
2022-08-18 17:13:59Zhi-Qi Cheng, Qi Dai, Siyao Li, Teruko Mitamura, Alexander HauptmannAbstract
Grounded Situation Recognition (GSR) aims to generate structured semantic summaries of images for ``human-like'' event understanding. Specifically, GSR task not only detects the salient activity verb (e.g. buying), but also predicts all corresponding semantic roles (e.g. agent and goods). Inspired by object detection and image captioning tasks, existing methods typically employ a two-stage framework: 1) detect the activity verb, and then 2) predict semantic roles based on the detected verb. Obviously, this illogical framework constitutes a huge obstacle to semantic understanding. First, pre-detecting verbs solely without semantic roles inevitably fails to distinguish many similar daily activities (e.g., offering and giving, buying and selling). Second, predicting semantic roles in a closed auto-regressive manner can hardly exploit the semantic relations among the verb and roles. To this end, in this paper we propose a novel two-stage framework that focuses on utilizing such bidirectional relations within verbs and roles. In the first stage, instead of pre-detecting the verb, we postpone the detection step and assume a pseudo label, where an intermediate representation for each corresponding semantic role is learned from images. In the second stage, we exploit transformer layers to unearth the potential semantic relations within both verbs and semantic roles. With the help of a set of support images, an alternate learning scheme is designed to simultaneously optimize the results: update the verb using nouns corresponding to the image, and update nouns using verbs from support images. Extensive experimental results on challenging SWiG benchmarks show that our renovated framework outperforms other state-of-the-art methods under various metrics.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08965
PDF
-
Stable Object Reorientation using Contact Plane Registration
2022-08-18 17:10:28Richard Li, Carlos Esteves, Ameesh Makadia, Pulkit AgrawalAbstract
We present a system for accurately predicting stable orientations for diverse rigid objects. We propose to overcome the critical issue of modelling multimodality in the space of rotations by using a conditional generative model to accurately classify contact surfaces. Our system is capable of operating from noisy and partially-observed pointcloud observations captured by real world depth cameras. Our method substantially outperforms the current state-of-the-art systems on a simulated stacking task requiring highly accurate rotations, and demonstrates strong sim2real zero-shot transfer results across a variety of unseen objects on a real world reorientation task. Project website: \url{this https URL}
Abstract (translated)
URL
https://arxiv.org/abs/2208.08962
PDF
-
Deploying Enhanced Speech Feature Decreased Audio Complaints at SVT Play VOD Service
2022-08-18 17:05:00Annika Bidner, Julia Lindberg, Olof Lindman, Kinga SkorupskaAbstract
At Public Service Broadcaster SVT in Sweden, background music and sounds in programs have for many years been one of the most common complaints from the viewers. The most sensitive group are people with hearing disabilities, but many others also find background sounds annoying. To address this problem SVT has added Enhanced Speech, a feature with lower background noise, to a number of TV programs in VOD service SVT Play. As a result, when the number of programs with the Enhanced Speech feature increased, the level of audio complaints to customer service decreased. The Enhanced Speech feature got the rating 8.3/10 in a survey with 86 participants. The rating for possible future usage was 9.0/10. In this article we describe this feature's design and development process, as well as its technical specification, limitations and future development opportunities.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08960
PDF
-
DeepClouds.ai: Deep learning enabled computationally cheap direct numerical simulations
2022-08-18 16:59:27Moumita Bhowmik, Manmeet Singh, Suryachandra Rao, Souvik PaulAbstract
Simulation of turbulent flows, especially at the edges of clouds in the atmosphere, is an inherently challenging task. Hitherto, the best possible computational method to perform such experiments is the Direct Numerical Simulation (DNS). DNS involves solving non-linear partial differential equations for fluid flows, also known as Navier-Stokes equations, on discretized grid boxes in a three-dimensional space. It is a valuable paradigm that has guided the numerical weather prediction models to compute rainfall formation. However, DNS cannot be performed for large domains of practical utility to the weather forecast community. Here, we introduce this http URL, a 3D-UNET that simulates the outputs of a rising cloud DNS experiment. The problem of increasing the domain size in DNS is addressed by mapping an inner 3D cube to the complete 3D cube from the output of the DNS discretized grid simulation. Our approach effectively captures turbulent flow dynamics without having to solve the complex dynamical core. The baseline shows that the deep learning-based simulation is comparable to the partial-differential equation-based model as measured by various score metrics. This framework can be used to further the science of turbulence and cloud flows by enabling simulations over large physical domains in the atmosphere. It would lead to cascading societal benefits by improved weather predictions via advanced parameterization schemes.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08956
PDF
-
Ered: Enhanced Text Representations with Entities and Descriptions
2022-08-18 16:51:16Qinghua Zhao, Shuai Ma, Yuxuan LeiAbstract
External knowledge,e.g., entities and entity descriptions, can help humans understand texts. Many works have been explored to include external knowledge in the pre-trained models. These methods, generally, design pre-training tasks and implicitly introduce knowledge by updating model weights, alternatively, use it straightforwardly together with the original text. Though effective, there are some limitations. On the one hand, it is implicit and only model weights are paid attention to, the pre-trained entity embeddings are ignored. On the other hand, entity descriptions may be lengthy, and inputting into the model together with the original text may distract the model's attention. This paper aims to explicitly include both entities and entity descriptions in the fine-tuning stage. First, the pre-trained entity embeddings are fused with the original text representation and updated by the backbone model layer by layer. Second, descriptions are represented by the knowledge module outside the backbone model, and each knowledge layer is selectively connected to one backbone layer for fusing. Third, two knowledge-related auxiliary tasks, i.e., entity/description enhancement and entity enhancement/pollution task, are designed to smooth the semantic gaps among evolved representations. We conducted experiments on four knowledge-oriented tasks and two common tasks, and the results achieved new state-of-the-art on several datasets. Besides, we conduct an ablation study to show that each module in our method is necessary. The code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08954
PDF
-
KDD CUP 2022 Wind Power Forecasting Team 88VIP Solution
Abstract
KDD CUP 2022 proposes a time-series forecasting task on spatial dynamic wind power dataset, in which the participants are required to predict the future generation given the historical context factors. The evaluation metrics contain RMSE and MAE. This paper describes the solution of Team 88VIP, which mainly comprises two types of models: a gradient boosting decision tree to memorize the basic data patterns and a recurrent neural network to capture the deep and latent probabilistic transitions. Ensembling these models contributes to tackle the fluctuation of wind power, and training submodels targets on the distinguished properties in heterogeneous timescales of forecasting, from minutes to days. In addition, feature engineering, imputation techniques and the design of offline evaluation are also described in details. The proposed solution achieves an overall online score of -45.213 in Phase 3.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08952
PDF
-
Towards Automated Process Planning and Mining
Abstract
AI Planning, Machine Learning and Process Mining have so far developed into separate research fields. At the same time, many interesting concepts and insights have been gained at the intersection of these areas in recent years. For example, the behavior of future processes is now comprehensively predicted with the aid of Machine Learning. For the practical application of these findings, however, it is also necessary not only to know the expected course, but also to give recommendations and hints for the achievement of goals, i.e. to carry out comprehensive process planning. At the same time, an adequate integration of the aforementioned research fields is still lacking. In this article, we present a research project in which researchers from the AI and BPM field work jointly together. Therefore, we discuss the overall research problem, the relevant fields of research and our overall research framework to automatically derive process models from executional process data, derive subsequent planning problems and conduct automated planning in order to adaptively plan and execute business processes using real-time forecasts.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08943
PDF
-
Sequence Prediction Under Missing Data : An RNN Approach Without Imputation
2022-08-18 16:09:12Soumen Pachal, Avinash AcharAbstract
Missing data scenarios are very common in ML applications in general and time-series/sequence applications are no exceptions. This paper pertains to a novel Recurrent Neural Network (RNN) based solution for sequence prediction under missing data. Our method is distinct from all existing approaches. It tries to encode the missingness patterns in the data directly without trying to impute data either before or during model building. Our encoding is lossless and achieves compression. It can be employed for both sequence classification and forecasting. We focus on forecasting here in a general context of multi-step prediction in presence of possible exogenous inputs. In particular, we propose novel variants of Encoder-Decoder (Seq2Seq) RNNs for this. The encoder here adopts the above mentioned pattern encoding, while at the decoder which has a different structure, multiple variants are feasible. We demonstrate the utility of our proposed architecture via multiple experiments on both single and multiple sequence (real) data-sets. We consider both scenarios where (i)data is naturally missing and (ii)data is synthetically masked.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08933
PDF
-
ManiFlow: Implicitly Representing Manifolds with Normalizing Flows
2022-08-18 16:07:59Janis Postels, Martin Danelljan, Luc Van Gool, Federico TombariAbstract
Normalizing Flows (NFs) are flexible explicit generative models that have been shown to accurately model complex real-world data distributions. However, their invertibility constraint imposes limitations on data distributions that reside on lower dimensional manifolds embedded in higher dimensional space. Practically, this shortcoming is often bypassed by adding noise to the data which impacts the quality of the generated samples. In contrast to prior work, we approach this problem by generating samples from the original data distribution given full knowledge about the perturbed distribution and the noise model. To this end, we establish that NFs trained on perturbed data implicitly represent the manifold in regions of maximum likelihood. Then, we propose an optimization objective that recovers the most likely point on the manifold given a sample from the perturbed distribution. Finally, we focus on 3D point clouds for which we utilize the explicit nature of NFs, i.e. surface normals extracted from the gradient of the log-likelihood and the log-likelihood itself, to apply Poisson surface reconstruction to refine generated point sets.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08932
PDF
-
Detecting Environmental Violations with Satellite Imagery in Near Real Time: Land Application under the Clean Water Act
2022-08-18 15:42:02Ben Chugg, Nicolas Rothbacher, Alex Feng, Xiaoqi Long, Daniel E. HoAbstract
This paper introduces a new, highly consequential setting for the use of computer vision for environmental sustainability. Concentrated Animal Feeding Operations (CAFOs) (aka intensive livestock farms or "factory farms") produce significant manure and pollution. Dumping manure in the winter months poses significant environmental risks and violates environmental law in many states. Yet the federal Environmental Protection Agency (EPA) and state agencies have relied primarily on self-reporting to monitor such instances of "land application." Our paper makes four contributions. First, we introduce the environmental, policy, and agricultural setting of CAFOs and land application. Second, we provide a new dataset of high-cadence (daily to weekly) 3m/pixel satellite imagery from 2018-20 for 330 CAFOs in Wisconsin with hand labeled instances of land application (n=57,697). Third, we develop an object detection model to predict land application and a system to perform inference in near real-time. We show that this system effectively appears to detect land application (PR AUC = 0.93) and we uncover several outlier facilities which appear to apply regularly and excessively. Last, we estimate the population prevalence of land application events in Winter 2021/22. We show that the prevalence of land application is much higher than what is self-reported by facilities. The system can be used by environmental regulators and interest groups, one of which piloted field visits based on this system this past winter. Overall, our application demonstrates the potential for AI-based computer vision systems to solve major problems in environmental compliance with near-daily imagery.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08919
PDF
-
Prompt Vision Transformer for Domain Generalization
2022-08-18 15:34:13Zangwei Zheng, Xiangyu Yue, Kai Wang, Yang YouAbstract
Though vision transformers (ViTs) have exhibited impressive ability for representation learning, we empirically find that they cannot generalize well to unseen domains with previous domain generalization algorithms. In this paper, we propose a novel approach DoPrompt based on prompt learning to embed the knowledge of source domains in domain prompts for target domain prediction. Specifically, domain prompts are prepended before ViT input tokens from the corresponding source domain. Each domain prompt learns domain-specific knowledge efficiently since it is optimized only for one domain. Meanwhile, we train a prompt adapter to produce a suitable prompt for each input image based on the learned source domain prompts. At test time, the adapted prompt generated by the prompt adapter can exploit the similarity between the feature of the out-of-domain image and source domains to properly integrate the source domain knowledge. Extensive experiments are conducted on four benchmark datasets. Our approach achieves 1.4% improvements in the averaged accuracy, which is 3.5 times the improvement of the state-of-the-art algorithm with a ViT backbone.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08914
PDF
-
NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
2022-08-18 15:11:24Zongrui Li, Qian Zheng, Feishi Wang, Boxin Shi, Gang Pan, Xudong JiangAbstract
Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light. Existing solutions alleviate the ambiguity by either explicitly associating reflectance to light conditions or resolving light conditions in a supervised manner. This paper establishes an implicit relation between light clues and light estimation and solves UPS in an unsupervised manner. The key idea is to represent the reflectance as four neural intrinsics fields, \ie, position, light, specular, and shadow, based on which the neural light field is implicitly associated with light clues of specular reflectance and cast shadow. The unsupervised, joint optimization of neural intrinsics fields can be free from training data bias as well as accumulating error, and fully exploits all observed pixel values for UPS. Our method achieves a superior performance advantage over state-of-the-art UPS methods on public and self-collected datasets, under regular and challenging setups. The code will be released soon.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08897
PDF
-
Pixel-Wise Prediction based Visual Odometry via Uncertainty Estimation
2022-08-18 15:09:39Hao-Wei Chen, Ting-Hsuan Liao, Hsuan-Kung Yang, Chun-Yi LeeAbstract
This paper introduces pixel-wise prediction based visual odometry (PWVO), which is a dense prediction task that evaluates the values of translation and rotation for every pixel in its input observations. PWVO employs uncertainty estimation to identify the noisy regions in the input observations, and adopts a selection mechanism to integrate pixel-wise predictions based on the estimated uncertainty maps to derive the final translation and rotation. In order to train PWVO in a comprehensive fashion, we further develop a data generation workflow for generating synthetic training data. The experimental results show that PWVO is able to deliver favorable results. In addition, our analyses validate the effectiveness of the designs adopted in PWVO, and demonstrate that the uncertainty maps estimated by PWVO is capable of capturing the noises in its input observations.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08892
PDF
-
Brand Celebrity Matching Model Based on Natural Language Processing
Abstract
Celebrity Endorsement is one of the most significant strategies in brand communication. Nowadays, more and more companies try to build a vivid characteristic for themselves. Therefore, their brand identity communications should accord with some characteristics as humans and regulations. However, the previous works mostly stop by assumptions, instead of proposing a specific way to perform matching between brands and celebrities. In this paper, we propose a brand celebrity matching model (BCM) based on Natural Language Processing (NLP) techniques. Given a brand and a celebrity, we firstly obtain some descriptive documents of them from the Internet, then summarize these documents, and finally calculate a matching degree between the brand and the celebrity to determine whether they are matched. According to the experimental result, our proposed model outperforms the best baselines with a 0.362 F1 score and 6.3% of accuracy, which indicates the effectiveness and application value of our model in the real-world scene. What's more, to our best knowledge, the proposed BCM model is the first work on using NLP to solve endorsement issues, so it can provide some novel research ideas and methodologies for the following works.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08887
PDF
-
Survey on Teleoperation Concepts for Automated Vehicles
2022-08-18 14:54:07Domagoj Majstorovic, Simon Hoffmann, Florian Pfab, Andreas Schimpe, Maria-Magdalena Wolf, Frank DiermeyerAbstract
In parallel with the advancement of Automated Driving (AD) functions, teleoperation has grown in popularity over recent years. By enabling remote operation of automated vehicles, teleoperation can be established as a reliable fallback solution for operational design domain limits and edge cases of AD functions. Over the years, a variety of different teleoperation concepts as to how a human operator can remotely support or substitute an AD function have been proposed in the literature. This paper presents the results of a literature survey on teleoperation concepts for road vehicles. Furthermore, due to the increasing interest within the industry, insights on patents and overall company activities in the field of teleoperation are presented.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08876
PDF
-
Robust Artificial Delay based Impedance Control of Robotic Manipulators with Uncertain Dynamics
2022-08-18 14:47:48Udayan Banerjee, Bhabani Shankar Dey, Indra Narayan Kar, Subir Kumar SahaAbstract
In this paper an artificial delay based impedance controller is proposed for robotic manipulators with uncertainty in dynamics. The control law unites the time delayed estimation (TDE) framework with a second order switching controller of super twisting algorithm (STA) type via a novel generalized filtered tracking error (GFTE). While time delayed estimation framework eliminates the need for accurate modelling of robot dynamics by estimating the uncertain robot dynamics and interaction forces from immediate past data of state and control effort, the second order switching control law in the outer loop provides robustness against the time delayed estimation (TDE) error that arises due to approximation of the manipulator dynamics. Thus, the proposed control law tries to establish a desired impedance model between the robot end effector variables i.e. force and motion in presence of uncertainties, both when it is encountering smooth contact forces and during free motion. Simulation results for a two link manipulator using the proposed controller along with convergence analysis are shown to validate the proposition.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08873
PDF
-
Lessons from a Space Lab -- An Image Acquisition Perspective
2022-08-18 14:44:40Leo Pauly, Michele Lynn Jamrozik, Miguel Ortiz Del Castillo, Olivia Borgue, Inder Pal Singh, Mohatashem Reyaz Makhdoomi, Olga-Orsalia Christidi-Loumpasefski, Vincent Gaudilliere, Carol Martinez, Arunkumar Rathinam, Andreas Hein, Miguel Olivares Mendez, Djamila AouadaAbstract
The use of Deep Learning (DL) algorithms has improved the performance of vision-based space applications in recent years. However, generating large amounts of annotated data for training these DL algorithms has proven challenging. While synthetically generated images can be used, the DL models trained on synthetic data are often susceptible to performance degradation, when tested in real-world environments. In this context, the Interdisciplinary Center of Security, Reliability and Trust (SnT) at the University of Luxembourg has developed the 'SnT Zero-G Lab', for training and validating vision-based space algorithms in conditions emulating real-world space environments. An important aspect of the SnT Zero-G Lab development was the equipment selection. From the lessons learned during the lab development, this article presents a systematic approach combining market survey and experimental analyses for equipment selection. In particular, the article focus on the image acquisition equipment in a space lab: background materials, cameras and illumination lamps. The results from the experiment analyses show that the market survey complimented by experimental analyses is required for effective equipment selection in a space lab development project.
Abstract (translated)
URL
https://arxiv.org/abs/2208.08865
PDF
