Do Transformers Need Three Projections? Systematic Study of QKV Variants
arXiv:2606.04032v1 Announce Type: new
Abstract: Transformers have become the standard solution for various AI tasks, with the query, key, and value (QKV) attention formulation playing a central role. However, the individual contribution of these three projections and the impact of omitting some rem...
Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning
arXiv:2606.04028v1 Announce Type: new
Abstract: The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning. These formats allow efficient and consistent representation of values in a small nu...
Pseudospectral Bounds for Transient Amplification in Coupled Gradient Descent
arXiv:2606.04031v1 Announce Type: new
Abstract: Coupled gradient descent--where the update of one parameter block depends on another--underlies bilevel optimization, two-time-scale stochastic approximation, and adversarial training. When the coupled Jacobian is block-triangular, asymptotic stabilit...
Testing the Test: Score-Direction Instability in Class-Split Anomaly Detection
arXiv:2606.02601v1 Announce Type: new
Abstract: Within-dataset class-split evaluation is widely used as a proxy for fully unconditional out-of-distribution anomaly detection. We show that this protocol can become ill-posed when the held-out anomaly class overlaps the normal mixture in representatio...
Assessing Region-Level EEG Contributions to Cognitive Workload Prediction
arXiv:2606.02598v1 Announce Type: new
Abstract: Accurate and generalizable estimation of cognitive workload from electroencephalography (EEG) is critical for human-centered and safety-critical systems. Although EEG is widely used for workload assessment, the consistency of region-level EEG contribu...
arXiv:2606.02597v1 Announce Type: new
Abstract: The development of brain-computer interfaces (BCIs) based on electroencephalograms (EEGs) has advanced significantly mainly to machine learning. Although the majority of earlier research has been on increasing classification accuracy, relatively littl...
Spectral Asymptotics of Neural Network Loss Landscapes: An Exact Decomposition of the Curvature Exponent
arXiv:2606.02596v1 Announce Type: new
Abstract: The curvature exponent $\alpha$ in $h_k \propto \sigma_k^\alpha$ -- governing how Hessian eigenvalues scale with gradient singular values -- varies systematically across layer types ($\alpha \approx 2$ for convolutions, $\approx 1$ for transformer att...
Optimal Transport-based Permutation-Invariant Bayesian Optimization of Offshore Wind Farm Layouts
arXiv:2606.00009v1 Announce Type: new
Abstract: Bayesian Optimization (BO) is widely and successfully adopted for solving optimization problems having an expensive-to-evaluate, black-box, and non-convex objective function. However, the vanilla BO algorithm is not able to exploit possible symmetries...
BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization
arXiv:2606.00079v1 Announce Type: new
Abstract: Mixture-of-Experts (MoE) large language models reduce per-token computation through sparse expert activation, but their deployment remains memory-intensive because all expert weights must be kept resident in memory. Existing MoE compression methods st...
Procedural Generation of First Person Shooter Maps using Map-Elites
arXiv:2605.30570v1 Announce Type: new
Abstract: We investigate the application of MAP-Elites (a well-known quality diversity algorithm) to design levels for First-Person Shooter (FPS) games. We consider two well-known map representations (All-Black and Grid-Graph) and introduce two novel representa...
Transforming and Encoding FTS for SAT Solving: What Helps, What Hurts (Extended Version)
arXiv:2605.30563v1 Announce Type: new
Abstract: Factored tasks are a classical planning representation that extends SAS+ with limited forms of disjunctive preconditions, conditional effects, and angelic nondeterminism. This allows for a more compact representation of tasks than traditional formalis...
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
arXiv:2605.30381v1 Announce Type: new
Abstract: Deceptive alignment, in which models maintain accurate internal representations while deliberately producing false outputs, remains a central challenge in AI safety. While strategic deception is the primary long-term concern, synthetic dishonesty - in...
LLMs Without Deep Neural Networks: New Architecture, Benefits and Case Study
arXiv:2605.30385v1 Announce Type: new
Abstract: The purpose of this article is to provide validation to my deep neural network alternative in the context of LLMs. Very recently, there has been a significant interest by Chinese researchers in a model called RBF network, as a substitute to standard D...
Review Arcade: On the Human Alignment and Gameability of LLM Reviews
arXiv:2605.28897v1 Announce Type: new
Abstract: LLM-generated reviews for scientific papers are gaining considerable traction and are even being officially piloted by major conferences. We have to assume that not only reviewers are using LLM-assistance, but also that authors use LLMs to revise thei...
One Mask to Rule Them All: On Hidden Facts after Editing and How to Find Them
arXiv:2605.28839v1 Announce Type: new
Abstract: Knowledge editing methods such as ROME and MEMIT update factual associations in transformer models by modifying MLP weights. While evaluated mainly by output behavior, their internal mechanism remains underexplored. We investigate whether edits rely o...
Mechanistic origins of catastrophic forgetting: why RL preserves circuits better than SFT?
arXiv:2605.28860v1 Announce Type: new
Abstract: Fine-tuning large language models (LLMs) frequently induces catastrophic forgetting of prior capabilities. Recent work has shown that reinforcement learning (RL) retains prior capabilities more effectively than supervised fine-tuning (SFT), attributin...
Behavior-Induced Mirror-Prox Temporal-Difference Learning for Faster Off-Policy Prediction
arXiv:2605.28849v1 Announce Type: new
Abstract: Gradient temporal-difference methods provide stable off-policy prediction with linear function approximation, but their practical performance is strongly affected by the geometry induced by the auxiliary-variable metric. Existing Mirror-Prox TD method...
Behavior-Aware Auxiliary Corrections for Off-Policy Temporal-Difference Prediction
arXiv:2605.28855v1 Announce Type: new
Abstract: Temporal-difference learning with function approximation can be unstable under off-policy sampling. TDC stabilizes off-policy TD through an auxiliary covariance correction, and TDRC further regularizes this correction in a single-timescale recursion. ...
Why LLMs Fail at Causal Discovery and How Interventional Agents Escape
arXiv:2605.27567v1 Announce Type: new
Abstract: Causal discovery is a cornerstone of scientific reasoning, yet whether large language models can perform it reliably remains an open question. Recent benchmarks show that even fine-tuned models plateau on simple causal graphs and degrade as complexity...
On the Origin of Synthetic Information by Means of Steganographic Inheritance
arXiv:2605.27551v1 Announce Type: new
Abstract: The origin of species has been the mystery of mysteries in natural science. By analogy, the origin of synthetic information, we suggest, is the mystery of mysteries in information science. The question carries a moral weight that a technical account c...
SilIF: Silhouette-Augmented Isolation Forest for Unsupervised Transaction Fraud Detection
arXiv:2605.26135v1 Announce Type: new
Abstract: Unsupervised anomaly detection is widely used in transaction fraud detection where labels are scarce. Isolation Forest (IF) is among the most popular classical methods due to its scalability and ease of deployment. We propose SilIF, an augmentation of...
CAFD: Concept-Aware DNN Fault Detection using VLMs
arXiv:2605.24008v1 Announce Type: new
Abstract: Fault detection for Deep Neural Networks (DNNs) has received increasing attention in recent years. While more advanced hybrid approaches have been proposed to combine multiple sources of information and outperform earlier techniques, they often incur ...
FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence
arXiv:2605.22868v1 Announce Type: new
Abstract: Autonomous systems and smart-industry deployments increasingly split computation across near-sensor, edge, and cloud resources, where tight energy, latency, and reliability budgets demand run-time adaptivity. In practice, deciding what to compute and ...