Search Results for "xml ui"

...It fills a gap in automation tooling by focusing on mobile-first workflows where traditional browser or desktop-based automation doesn’t work; such as logistics, gig work, field operations, and other industries reliant on phones or tablets. The project works by using Android’s accessibility API to extract structured UI state (as XML) from the device, which is then fed to a large language model (LLM) like OpenAI’s models for decision-making, and actions are executed via the Android Debug Bridge (ADB). This approach bypasses expensive vision-based models and provides faster, cheaper automation with fine-grained interaction capabilities (for example, tapping buttons, typing text, navigating screens).

This product recognizes oral commands and translates them to domotic orders for a domotic system. This product does not implement a domotic system. This product is an interface to be plugged to a domotic system. The speech recognition is done by an arduino UNO board and an EasyVR shield. Available oral commands are generated from a house description file in XML format. The oral commands have to be trained for a specific users. For this purpose 2 interfaces are provided: a command line...