Moqui Framework

Moqui Framework Release Notes =========================================================================== Release 1.0.1 - 31 December 2011 =========================================================================== Version 1.0.1 is primarily a maintenance and bug fix release. The few new features are mostly extensions to existing features for improved flexibility, to round out functionality, and better support standards. This release includes: - All the great features of the 1.0.0 and previous releases - Updated Gradle build file for version 1.0-milestone-6 (previously ran on 1.0-milestone-3) - Added support to automatically clear caches for view-entity based in updates to member-entity records, as long as all member-entity PK fields are directly or indirectly aliased on the view-entity - Improved entity-auto service error handling - Changed login screen to render in the same request instead of a redirect and to return a status code of 401 (related to bug #3442045) - Made form field sub-fields more flexible, can include any widget tag under them now; this has been tested with render-mode and should work with others as they just run the corresponding macro for the widget element, but your mileage may vary - Added support for a classes directory under the runtime directory, and under each component directory to go along with the lib directories so that classpath resources do not have to be in a jar file - Added support for JSR-223 (javax.script) compatible scripting languages through configuration only (in moqui-conf file) so no ScriptRunner interface implementation is needed for these - Added support for JavaScript (.js) server-side scripts through javax.script interface - Added support for StartTLS and SSL to sendEmailTemplate service, based on the configuration in the EmailServer record Bug Fixes: - Removed emailTypeEnumId from the PK of the EmailMessage entity, it should not have been there and is not needed (or wanted); this PK change will require modifying or rebuilding that table in existing systems - Fixed issue with handling entities in different packages that have the same name - Instead of submit buttons always having name=submit, they now use the name of the XML Form field; to see if one button or another was pressed just check to see if the parameter was passed; also changed all submit field names to submitButton to avoid conflict with standard DOM function "submit" for form elements (thanks to Vasanth for reporting this) - Fixed bad reference to ComparisonOperator in DateCondition class - Fixed issue with settings nulls in an entity-auto update service or anywhere the dbValueMap is not populated from the database - Fixed bug (#3442014) with funny FTL eval of label.@encode attribute - Fixed issue with ResourceFacade classpath:// URLs that were not using thread's context ClassLoader, and so weren't finding classpath entries in runtime or component lib and classes directories - Fixed issue with ResourceReference, TemplateRenderer, ScriptRunner, ServiceRunner, and Java service implementation class loading to use the current Thread's context ClassLoader which will get the component and runtime lib and classes resources - Fixed issue where automatic reverse relationship did not work for full entity names - Fixed issue with findRelated and findRelatedOne when forUpdate is null - Fixed bug in StupidUtilities.basicConvert so that String to Boolean conversions do not use the default groovy non-empty rule for true, ie so that the string "false" will actually evaluate to false - Fixed service parameters validation to not treat false boolean values as empty values (ie we can't use groovy non-empty definition for Booleans) - Fixed permission issue in resetPassword service =========================================================================== Release 1.0.0 - 21 November 2011 =========================================================================== Here it is, the initial production release of the 1.0 series. This release includes various fixes and new features to facilitate easy use based on more extensive testing and real-world development using the framework. More examples of different patterns exist in Moqui Framework itself, as well as in add-on projects including Mantle, POP Commerce, and so on. This release includes: - All the great features of the preview, beta and rc1 releases - For field name pattern consistency changed GeoAssoc.geoIdTo to toGeoId, StatusValidChange.statusIdTo to toStatusId, and UomConversion.uomIdTo to toUomId - Added support to skip stats tracking based on a conditional expression - More performance profiling and improvements - entity-auto update and delete services no longer do a find first - various frequently used objects are cached in UserFacade, especially those used for authz and tarpit, making many things faster and cached queries around 4 times faster than the already optimized speed - many small performance improvements, changes to get around Groovy and Java quirks - Change components to load in reverse of order initialized (later components override earlier ones) - Increased text field sizes (short to 63, medium to 255, long to 4095) - Changed a bunch of fields from text-long to text-medium that are either primary-key fields, or that don't need to be so long, or in entities where there are too many text-long fields - User Facade - Added support for login via HTTP Basic Authorization, mostly for things like RESTful services and support for clients using it - Entity Facade - Allow (optional) reference of entity by package-name + entity-name, - Change framework to reference all its entities by fully qualified name - Allow separator (hash) between title and related-entity-name in relationship name now that related-entity-name may be packages and such, and looks funny - Primary sequenced ID banking for better performance when creating records with sequenced single field primary key (default bank size 50) - Change lastUpdatedStamp to tx timestamp to sync/export/etc data without splitting transactions - Supports seed-data element in entity and extend-entity elements so that seed data can be defined and managed along with entity definitions - Option to disable Entity ECAs during data load - Isolate cache clearing failures, don't ever kill calling method (ie never throw an exception/throwable) - Screen Facade - Filter menu rendering by authorization - List multi form submit button below and not in own column - Allow field, auto-fields-entity, and auto-fields-service to appear in any order and preserve that order when exploding the form definition - Auto Screen move Edit button on list forms to left side - If form transition has a single service (service-call instead of actions elements) then automatically use that service for validation information for fields with matching name - Improved handling of context and error parameters to more cleanly and easily repopulate forms after a server-side error - Example (/echopath) of handling "path info" beyond screen or transition in request path - Added support for other types of wiki files including MediaWiki, Textile, TracWiki, and TWiki (in addition to Confluence) =========================================================================== Release 1.0-rc1 - 3 August 2011 =========================================================================== The first 1.0 release candidate of the Moqui Framework (version 1.0-rc1) was released on 3 August 2011. This release includes various small new features based on feedback and actual use, including Apache Shiro for security (authc and authz), build using Gradle which supports Maven repositories and with directory layout changed to follow Maven conventions, the ability to run arbitrary SQL and get the results back in an EntityListIterator, and the Data View tool that allows users to build queries and data exports on the fly. This release candidate also includes improvements based on more testing, including performance testing and profiling to dramatically increase the speed of various operations, especially those done frequently. As a release candidate from this point only minor changes are expected before the 1.0 final production-ready release. This release includes: - All the great features of the preview releases, 1.0-beta1, and 1.0-beta2 - More testing and bug fixes - Various performance improvements based on profiling (thanks to preview licenses for YourKit and JProfiler) - Updated all libraries with updates available as of 15 July 2011 - Changed build to use Gradle instead of Ant - Support for multiple root screens determined by host name using regexp, or in other words virtual host support - UserPreference records refined so that keys can be defined with Enumeration records, but don't have to be - Some changes to better support the Mantle UDM (Universal Data Model), including better automatic foreign key names, looking for a "mantle" directory under the runtime directory where the mantle components can go (can actually have the GIT checkout of mantle right there) - Added a UserGroup (ALL_USERS) that all users are automatically members of - Security: tightened permissions, using the entity.@authorize-skip much less by adding disable/enableAuthz calls to framework code, and adding an authorization for users to view their own UserAccount record. - Directory structure and build changes to simplify and be more consistent with de-facto standards from Maven/Gradle/etc - Small changes to make it easier to include the runtime directory inside the deployed war file. - Resource Facade - Added ScriptRunner interface to allow for easily adding scripting languages to be supported by the ResourceFacade - User Facade - Incorporated Apache Shiro, which is now used for all authentication in Moqui, by default through the MoquiShiroRealm and other realms such as for LDAP or ActiveDirectory can be configured with shiro.ini - Permission model and checking for simple/flat permissions (to be used as an alternative to artifact authz) - Run permission and role (UserGroup) checks through Shiro realm - Entity Facade - Manual SQL find with results mapped to entity with results returned in an EntityListIterator (EntityFacade.sqlFind()) - Database-driven view entity - view-entity: merge the view-link element into the member element (also for DynamicViewEntity, DbViewEntity, *DbView screens) - Service Facade - Added entity-auto support for the verb "store" which does a create if record does not exist and update if it does - Support of pre-configured scheduled job exists, see: http://www.quartz-scheduler.org/docs/cookbook/JobInitPlugin.html - Artifact Execution Facade - Persist data about authz failures (in ArtifactAuthzFailure entity) - Artifact Tar-pitting based on configuration in db (see example moqui.security.ArtifactTarpit in ExampleSecurityData.xml) - Support authorization checks through Shiro realm - Tools Screens - Auto Screen - Order by links in column headers by default (by default for all auto fields entity as well) - Data View - Choose master entity, select fields from all related entities, save as DbViewEntity and related records - Specify functions for selected fields - Choose column(s) to order by - Specify search constraints - View in webapp - Export to CSV - Service - Select and run service from form based on service definition =========================================================================== Release 1.0-beta2 - 30 May 2011 =========================================================================== The 1.0-beta2 release includes a number of new tools, most notably the Auto Screen UI that creates a user interface based on entity and relationship definitions for master entities, and tabs for the detail and associated entities of each. This release also includes a number of smaller new features and bug fixes found while building a variety of functionality with Moqui Framework. One notable example of that is automatic or explicit alternate screen selection based on the HTTP request User-Agent header, and there is an example of alternate screens for the iPad. Overall release 1.0-beta2 is more robust, functional, and reliable than the previous release. This release includes: - All the great features of releases 1.0-preview1 to 1.0-beta1 - Changed so that web.requestParameters Map is automatically added to the context when it is initialized for the web (helps reduce dependency in code on web-specific stuff, ie ec.web.requestParameter references) - L10nFacade: moved number and date/time parsing and formatting methods from UserFacade to L10nFacade - TransactionFacade: changed suspend() and resume() methods to better use internal stack of suspended transactions for consistency and a simpler API; also added stack of Exception to track where transactions are suspended for easier finding of lock wait timeout types of errors - Moved MoquiDefaultConf.xml from framework/api to framework/impl since it contains a lot of implementation-specific configuration - Updated Atomikos TransactionsEssentials to 3.7.0 and ehcache to 2.4.1 - Support <component>/lib and runtime/lib jar loading in a cached and extended/extensible ClassLoader - Screen transition now supports restriction by HTTP request method - Trim stack traces to not include Groovy proxy/etc calls (Sam's idea); implemented for now in BaseException, will use in other areas over time - Changed Example entities and seed data to follow pattern to be used in Moqui and Mantle where the relationship.title is the enumTypeId or the statusTypeId - XML Screens - Dynamic transition name handling (support regex transition names) - Specify macro template in screen def to override default in moqui-conf - Template for xsl-fo render mode - Added support for drop-down.entity-options in Screen Form - Auto create drop-downs for fields with a relationship to Enumeration using relationship.title as the enumTypeId, check to see if is valid enumTypeId before creating drop-down - Auto create drop-downs for fields with a type one relationship, with limit of 200 to avoid crazy drop-downs - Choose default subscreen (override by path) based on user agent or other condition (using any data available in the ec) - Override login path in screen (use value from deepest screen in path) - Resource Facade - Method for FOP transform to PDF, etc - Support .gstring templates (groovy.codehaus.org/Groovy+Templates) - Entity Facade - More intelligent group by fields, ie add all fields in view-entity and DynamicViewEntity to group by list if they don't have a function; with support for this get rid of alias.@group-by attribute, not needed when done automatically - Entity Master data export (master record and all dependent records) - Entity and Dependents export: if an entity is a dependent of another, and of the master then export it after the other entity it is a dependent of - Tools Screens - Auto Screen - Master Entity List (entities with dependents) - AutoFind Screen (find, create, delete master entity records) - AutoEditMaster (tab to edit master entity record) - AutoEditDetail (tab for each dependent entity with CrUD) - Entity - Entity Data UI - entity list - find on entity with delete - create/update entity record - view entity record with related entities and links to them - Entity Data Import screen - Entity Data Export screen =========================================================================== Release 1.0-beta1 - 01 Apr 2011 =========================================================================== The 1.0-beta1 release is the first feature-complete release for the 1.0 series. Various tools screens that may be included in the 1.0-final release are not yet complete, but the framework itself is feature-complete. Following this release the focus for the framework shifts from implementation to testing and to building applications and tools based on it (in Moqui Framework, Moqui Mantle, and in other Moqui-based projects). This release includes: - All the great features of releases 1.0-preview1 to 1.0-preview3 - Changed UserAccount so that userId is internal/sequenced and a new username field is used for auth/login - Seed data changes to follow the Mantle pattern where enumTypeId is the same as entity.relationship.@title for Enumerations - XML Screens - Screen form-list form-list-column support - Screen form-single field-layout support - Screen form-list pagination (defaults in search-form-inputs) - Template for csv render mode (mainly for form-list report-like output) - Template for text render mode (for email alternative message, etc) - Template for xml render mode (like CSV, but more structured and support multiple forms per screen) - Support URL parameters for renderMode and pageNoLimit which along with the lastStandalone parameter make it easy to reuse screens and forms meant for user interaction as definitions for CSV output - WebApp event actions for after-startup, before-shutdown - Service Facade - Incoming email handling with Email ECA Rules (emeca) with the org.moqui.impl.EmailServices.pollEmailServer service, configured with the EmailServer entity - Service (org.moqui.impl.EmailServices.sendEmailTemplate) to send email based on settings in the EmailTemplate entity - Entity Facade - Automatic reverse-many relationship for type one relationships coming the other way (done on-demand at run-time) - Password reset email and forgot password form - Some fixes so that Moqui will run in Java on Windows - Testing and fixes for EntityFindImpl.count(); screen.transition.*-response parameters; screen.transition with no actions or conditional responses to act as an alias, even with parameters/etc; script type services; updatePassword service and various security/password settings =========================================================================== Release 1.0-preview3 - 19 Mar 2011 =========================================================================== Moqui 1.0-preview3 introduces various features to automatically and easily make the applications you build using the previously available tools easier to deploy and administer, and easier to test and get information about what is happening as artifacts are executed. These include multi-tenant support, artifact hit and hit bin tracking (for screens, transitions, services, and entities), database-driven artifact authorization with many advanced options, XML-RPC and JSON-RPC calls for Moqui services, and Tools screens to administer and get information about various parts of the framework. NOTE: just before this release the official code repository for Moqui moved from SourceForge SVN to Git on GitHub so that users can more easily create Git forks for their own use, and to allows pulls from their Git forks back into the main Moqui repository. The Moqui GitHub URL is: https://github.com/jonesde/moqui This release includes: - All of the great features of 1.0-preview1 and 1.0-preview2 - Updated various libraries included (esp Groovy, ehCache, Quartz) - Multi-tenant support (shared app server, separate databases) - Finished impl and test for localized messages and entity fields - WebApp event actions for first-hit-in-visit, before-request, after-request, after-login, before-logout - JSON-RPC 2.0 incoming service handler (with Dzhuvinov JSON-RPC) - JSON-RPC 2.0 outgoing service runner (with Dzhuvinov JSON-RPC) - XML-RPC incoming service handler (with Redstone XML-RPC) - XML-RPC outgoing service runner (with Redstone XML-RPC) - XML Screens - Screen form client JS validation based on target transition service definition (define once, run on client and server); using jquery validation (http://docs.jquery.com/Plugins/Validation); so far for required, number-integer, number-decimal, text-email, text-url, text-digits, credit-card; try out on the Edit Example screen - Screen form-list multi=false support (one form per row) - Screen form-list multi=true support, service call multi=true - Screen form-list support form-links inside main form - Screen form auto-complete for text-line - Screen subscreen menu dynamic tabs (last level only) that loads content as a standalone screen from the server with an ajax call - Screen form focus-field, skip-start and skip-end supported - Visitor tracking - ArtifactHit and ArtifactHitBin tracking for screen, screen-content, transition, service and entity (bin only) - User preference handling - Artifact Execution Facade - ArtifactExecutionFacade for artifact stack tracking - Artifact Authorization based on configuration in db - Call ArtifactExecutionFacade.push(aei) (to track, check authz, etc) for screen, transition, service, entity - Disable authz on data loading; command line and assumed to be safe - Handle direct service/entity calls, ArtifactGroupMember with pattern - Review and test to determine which entities can skip authz for view and create (ie some view only, some create only, some view and create) - For entity-implicit service calls just look for entity authz - Support authorization on subscreens even if no authz on parent screens, ie authz for EditExample even though not for ExampleApp or any other screens in that app - Record-level authorization (ArtifactAuthzRecord/Cond) - Authorization with a service (ArtifactAuthzService) - Tools Screens - Artifact Hit Bins - Audit Log - Visit and Hit Info/Stats (Find Visits, Visit details with hit list) - Cache List and Elements - Localization: Messages, Entity Fields - Simplified sequenced ID API for both primary and secondary sequenced IDs and moved configuration to entity definition, and methods to EntityValue - Cleanups for xml-actions elements: from-field to from, call-service to service-call, call-script to script; better now than later to make the XML a bit cleaner and more clear (for example from-field can be any groovy expression and the -field implies otherwise) - Significantly more testing and bug fixes related to synchronization, performance (WAY faster, but more room for improvement), transactions, style, etc =========================================================================== Release 1.0-preview2 - 01 Mar 2011 =========================================================================== With release 1.0-preview2 the Moqui Framework is now nearly complete and has sufficient functionality to build a wide variety of functional and useful enterprise, or less formal, applications. This release includes a functional example application with various screens demonstrating features of the framework and recommended practices and patterns to use when developing with Moqui. While not feature-complete for the planned 1.0 feature set, the 1.0-preview2 release is ready for early adopters. For those interested, this is a great time to try out Moqui for a pet project or even a more formal project that is just getting started and won't be deployed in production for at least a few months. Feedback on functionality and bugs by early adopters would be extremely helpful for the project at this point, both to validate (or invalidate...) designs and to more extensively test the implementation. This release includes: - All of the great features of 1.0-preview1 - Significant updates to Example application screens (most now working) - Added various configuration options and examples for flexible deployment and overriding templates for screen macros, xml-actions, etc - Executable WAR file is more self-contained, ie less required in runtime but with a few more override options in runtime too - Root web screen split to support undecorated subscreens and apps - XML Screens - Screen form elements, all implemented (including file upload, *-find) - Standalone screens that are rendered independently of parent screens and can still be subscreens of any other screen - Screen pre-actions that all run before any parent screen is rendered to allow for setting parameters in parent screens, setting up general things, etc - Screen single-form HTML uses fieldset and label instead of a table - Screen form header-field widget rendering - Screen form header-field.@show-order-by support - Screen form localized output formatting, based on service parameter format if applicable, otherwise on format specified in form field - Output encoding, configurable on various screen/form elements - I18n (with L10nFacade) for labels and titles in screens, forms, menus - Added jquery and jquery-ui and the smoothness theme - Added container-dialog widget that creates a button that when clicked on opens a dialog with the contents of the container-dialog. - Added datetimepicker, which is the standard jquery datepicker plus the timepicker add-on, combined using some code from Apache OFBiz - Extended and now using database-driven screen visual themes - Can send Maps to HTTP response as JSON string (Web Facade), example of this is in EditExample.xml - Service Facade - Service expand auto-parameters, get parameters from implements - Finished implementation of async/scheduled service calls - Service run on commit and on rollback of current tx - Service parameter type/subtype checking - Service parameter ESAPI/Antisamy HTML checks (any/safe/none) - Service authentication - Service parameter conversion/parsing and validation checks - Service ECA rules - Entity Facade - View-entity fixes and testing, now most view-entity options work - Support entity-find.search-form-inputs element for form *-find fields - Automatic cache clearing for one, list (using RA cache), count caches - Entity extension - Entity audit-logging - Entity field encryption - Entity ECA rules - Support offset and limit as LIMIT/OFFSET or OFFSET/FETCH depending on configuration on the database element in the Moqui conf file - Database meta-data support for adding indexes when tables added - Automatically add columns if missing in table check - Add foreign keys to existing tables when table created - Added EntityDbMeta.createForeignKeysForExistingTables() to check the foreign keys of all entities with existing tables and add missing ones where both tables exist; useful to run once all desired tables have been created (through a data load, test script, etc) to round out the DB meta-data =========================================================================== Release 1.0-preview1 - 09 Feb 2011 =========================================================================== Release 1.0-preview1 is the first preview release of the Moqui Framework. This first release includes a number of complete tools, implementation of most of the Java API, and support for many of the configuration elements and the various XML files. There are many things left to do before all 1.0 features are implemented and you'll see various "TODO" comments scattered around, though not every feature yet to be implemented is represented by such a comment. This release includes: - Flexible deployment - Executable WAR file for command-line data loading and embedded Winstone servlet container (Derby DB, Atomikos JTA&CP, other required JARs) - The same WAR file can be dropped in a servlet container like Tomcat or Jetty, or a more general app server like JBoss or Weblogic - Supports Derby, HSQL, MySQL, Oracle and Postgres OOTB, support for additional databases primarily by configuration - Separate runtime directory with configuration, components, logs, database, JCR repo, etc; is the same no matter how the WAR is deployed - Use the supplied MoquiServlet, or write your own based on that example - Everything runs from an instance of the ExecutionContextFactory, so it is easy to include in custom programs or deploy through OSGi, etc - Specify runtime directory path and conf file location using properties file or System properties (java -D arguments) - XML Screens - XML Screen rendering for HTML - Screen menus and sub-screens - Hierarchical to any depth, URL is path to screen in hierarchy - URL building based on configuration - Sub-screens by directory structure, XML elements, db record - Screen sub-content (stand-alone, included in screen) including support for FTL and Confluence Wiki templates, and text and binary files - Screens and sub-content can be located in files or in a JCR repository - Can include or inline HTML and other text content - Visit tracking - Execution Context & General Facades - WebFacade for access to Servlet objects, parameter maps, etc - UserFacade for current user data, login/logout/authentication, etc - MessageFacade for general and error message management - L10nFacade for localization of text - Resource Facade for access to classpath, file, JCR, etc resources, and for running scripts and rendering templates (with caching for all); - Interface to implement for any type of template rendering desired - Interface to implement for any type of resource reference and access - LoggerFacade for general logging, especially for non-class code - CacheFacade for general caching, backed by ehcache - TransactionFacade for general JTA-type operations and tracking what is currently going on with transactions, where errors originated, etc - ScreenFacade for rendering screens using the ScreenRender interface - Service Facade - Service runners for Java, XML Actions, inline actions, Scripts - Entity-auto runner for CrUD ops without even defining the service - Interface for adding your own service runners - Can call sync, async or scheduled (using Quartz Scheduler) - Entity Facade - EntityValue with support for create, update, delete, refresh db operations, find related, various get and set options - EntityFind with various conditions, etc supported and results of one, list, iterator, count, updateAll, or deleteAll. - Comprehensive data loader with API and command-line access - Database meta-data support for automatically creating tables at runtime as they are used - Primary and secondary sequenced ID generation - Security - ESAPI input canonicalization/filtering - User account and flexible password constraints and management - Authentication in XML Screens - Can require encryption in XML Screens - XSRF protection by requiring encryption of input in body parameters - XML Actions - Converts to Groovy code then compiles for good runtime performance - Embeddable in service definitions, screens, etc - Example application with screens, sub-content (CSS, html.ftl, cwiki.ftl), entities and services, demo app and security and l10n data, etc - Sample configurations for development, staging, and production - Configurable "root" screen with sample HTML wrappers, login, menus, etc =========================================================================== Features designed or considered, but not to be included in 1.0 series =========================================================================== - XML Screens - Screen tree elements - Screen section-iterate pagination - Screen form automatic client JS validation for service in-parameters for matches, number-range, text-length, text-letters, time-range, credit-card.@types - Dynamic Screens (database-driven: DynamicScreen* entities) - Dynamic Forms (database-driven: DynForm* entities) (see Drupal CCK for some similar concepts; focus on extend/override/etc of existing forms) - Form render template for cwiki render mode (even useful?) - Service Facade - Service database-driven semaphores - Service runners for proxy service calls to another Moqui server via HTTP, JMS, and RMI (TX enlisted), email, XMPP (any of these useful, or generally better to use Mule or ServiceMix/Camel?) - Entity Facade - Support for CouchDB, and perhaps other NoSQL databases - LiquiBase integration, entity.change-set element - EntitySync tool - Entity complex aliases in views (EntityDefinition:246) - View entity view-link.econdition* support (EntityFindBuilder:196) - Artifact Execution Facade - Call ArtifactExecutionFacade.push() (to track, check authz, etc) for other types of artifacts (if/as determined to be helpful), including: Component, Webapp, Screen Section, Screen Form, Screen Form Field, Template, Script, Entity Field - Tools Screens - Auto Screen - Editable data grid, created by form-list, for detail and assoc related entities - Entity - SQL Query Run/View - Entity model internal check (relationship, view-link.key-map, ?) - Database meta-data check/report against entity definitions; NOTE: use LiquiBase for this - Script Run (or groovy shell?) - Service - Configure and run chain of services (dynamic wizard) - Scheduler (view scheduled services with cancel and add) - Artifact Info screens (with in/out references for all)