0.17.1

[Beam]

• Added caching of colors order in dashboard (order is first assigned according to predefined order and number of traffic per each referer medium). remp/remp#719
• Added missing IE11 polyfill to support Promise in IE11. remp/remp#795
• Added support to track IDs of user's subscriptions granting access to content. remp/analytika#11
• Added optional env variable COMMANDS_OVERLAPPING_EXPIRES_AT which controls overlaping expiration and changed default commands without overlaping expiration to 15 minutes instead of 24 hours. remp/remp#768

[Campaign]

• Added Newsletter rectangle banner type to allow subscription to newsletter directly from within the banner. Further configuration and backend proxy is necessary to use the feature. Refer to README and .env.example for additional info. remp/remp#618
• Added missing IE11 polyfill to support Promise in IE11. remp/remp#795
• Added optional env variable COMMANDS_OVERLAPPING_EXPIRES_AT which controls overlaping expiration and changed default commands without overlaping expiration to 15 minutes instead of 24 hours. remp/remp#768
• Moved caching of SegmentAggregator (required in showtime.php) to HTTP middleware and CampaignsRefreshCache command. Done to avoid caching when running composer install and its hooks (such as artisan package:discover). remp/remp#798

[Sso]

• Added optional env variable COMMANDS_OVERLAPPING_EXPIRES_AT which controls overlaping expiration and changed default commands without overlaping expiration to 15 minutes instead of 24 hours. remp/remp#768

[Mailer]

• Fixed issue with Twitter embeds caused by the external library. remp/remp#796

[Mailer]

• Added option to show mail template by code in TemplatePresenter->showByCode method. remp/crm#1626

0.17.0

Important

• BREAKING: All REMP applications now primarily use rtm parameters (our replacement for utm parameters, to avoid conflicts with other tracking software).
  When deploying this release, you have to deploy the Segments/Journal app first and the Tracker app second. Everything else (Beam, Campaign, Mailer, SSO) has to be deployed afterwards (the order doesn't matter). This is due to the internal change in how Tracker stores rtm parameters in underlying Elasticsearch storage.
• BREAKING: Segments/Journal app aggregations using utm parameters (namely utm_campaign, utm_content, utm_medium, utm_source and banner_variant) has to be rewritten to use rtm parameters (rtm_campaign, rtm_content, rtm_medium, rtm_source and rtm_variant). If one wants to aggregate old data (with utm_ parameters) together with new data, it has to be done in two separate calls to Segments API (one with utm_ and another one with rtm_ aggregation parameters).

Project

• Fixed PHP 7.4 docker image build dependencies.

[Mailer]

• Fixed issues with settings page caused by internal config names renaming. remp/crm#1616
• Fixed main search issue due to the internal changes in the past release. remp/remp#786
• Improved search speed of the main search bar. remp/remp#786
• Fixed bug on job edit form causing error when loading the form.
• Fixed conversion stats processing if there are no templates/batches to process.
• Fixed rendering issues of 4xx pages when invalid page was requested.
• Fixed conversion stats processing type-related issues.
• Fixed mail_layout_id type-related issues in /api/v1/mailers/templates API.

[Beam]

• BREAKING: API endpoints to get "top" tags or authors now utilize their external_id to filter the data. Please make sure you already use /api/v2/articles/upsert to populate article information.
• Fixed scenario in JS library when fallback cookie value expiration was not updated with the main local_storage expiration.
• [Tracker]: Added new parameter commerce_session_id to track/commerce endpoint of API tracker. remp/crm#1559
• Added new identifier commerce_session_id into remplib.js to identify unique commerce process. remp/crm#1559
• [Segments]: UTM to RTM parameters transition. remp/remp#779
• [Tracker]: UTM to RTM parameters transition. remp/remp#779
• Removed statically set memory_limit configuration within some memory-extensive commands. remp/remp#788
• Added support for configurable memory limit for each command via COMMANDS_MEMORY_LIMITS environment variable. See .env.example for more information. remp/remp#788
• Improved dashboard graph - snapshots of concurrent data are mapped to fixed time points to avoid displaying glitches in graph. remp/remp#763
• Changed scheduled aggregation and snapshotting command calls to be non-blocking (so they don't wait for each other to be executed). remp/remp#763
• Command composer install now works without DB connection (removed check for migrations table)

[Campaign]

• Command composer install now works without DB connection (removed check for migrations table)

0.16.0

Project

• Removed --no-bin-links switch from `Docker/php/remp.sh installation script. Bin symlinks are required after the latest changes in Yarn package commands.
• Added PHP 7.4 syntax check to .gitlab-ci.yml.

[Beam]

• Added caching to Tracker preflight requests to limit number of OPTIONS calls. Cache is now set to 1 hour (3600 seconds) and it effectively adds Access-Control-Max-Age: 3600 header to preflight responses.
• Fixed issue with slow pageview processing queries due to string/int type conflict in the query parameters. remp/remp#766
• [Segments]: Fixed search.max_buckets Elastic issue when aggregating too big chunk of pageviews data. Changed internal implementation of Segments API, aggregation is now using Elastic composite aggregation and pagination. remp/remp#662
• [Segments]: Fixed return type of count histogram items. Float was changed to int as count can always return integers anyway. remp/remp#622
• [Segments]: Removed offset parameter for histograms. Beam APIs haven't used it and composite index (new Elastic-7-friendly implementation) doesn't support it yet. remp/remp#622
• Added filter by content type to conversion, article pageviews, article conversions and author detail listing. remp/remp#769
• Added ability to compute section stats segments using ComputeSectionSegments command + added configuration category and items for this feature + added test configuration screen which uses command to generate results and sends them to specified email address. remp/remp#424
• Added run ComputeSectionSegments command to laravel console Kernel. remp/remp#424

[Campaign]

• Allowed search and paging in dashboard schedules table. remp/remp#755
• Fixed possibility of banner close buttons being overlayed by Safari scrollbar. remp/remp#764
• Extended banner close button area for easier manipulation on touch-based devices. remp/remp#764
• Fixed initialization of REMP JS library (may have caused a bug when banner aimed to anonymous users was shown to a logged-in user). remp/crm#651
• Fixed invalid logger access in Showtime request causing fatal errors if user's adblock information was missing. remp/remp#774

[Mailer]

• BREAKING: Some portion of classes was not PSR-0/PSR-4 compliant and were moved to their correct folders/namespaces. Please update your config.local.neon based on the updated example file.
• BREAKING: Updated league/event from version 2 to 3 - there is config change - please change League\Event\Emitter to League\Event\EventDispatcher
• BREAKING: Removed Remp\MailerModule\Console\Application and keep only native Symfony Application. Change in config -> use add instead of register
• BREAKING: Updated Twig library to version 3. Also introduced IEngine for templates rendering. More detail here
• BREAKING: Replaced Remp\MailerModule\Replace\ReplaceInterface with Remp\MailerModule\ContentGenerator\GeneratorInput\IReplace. Please see the example implementations and update your implementation accordingly.
• BREAKING: Added php types into most missing places (interfaces change). Please run make phpstan after update to check your custom changes.
• BREAKING: Unified usage of ActiveRow, IRow and DateTime (interfaces change). Please run make phpstan after update to check your custom changes.
• Added declare(strict_types=1); to all php files.
• Fixed possible issue with asset location on Mac when Valet is used for development.
• Upgraded vlucas/phpdotenv from version 2 to 5 and removed usage of getenv() function.
• Removed obsolete deploy script.
• Fixed Hermes worker RedisDriver not restarting if there are no new tasks to handle. remp/crm#1561
• Refactored bootstrap file to follow new Nette skeleton structure.
• Added missing user_id index to mail_user_subscriptions table.
• Updated monolog/monolog from version 1 to 2
• Updated mailgun/mailgun-php from version 2 to 3
• Updated robmorgan/phinx to 0.12.
• Updated phpunit/phpunit from version 7 to 9.
• Fixed incorrect unsubscribed stats for emails sent directly to users (not via jobs). remp/remp#771
• Fixed crash in mail:process-job command when article meta fetch fails. remp/remp#773
• Added graph of unsubscribed users in time to mail type detail. remp/remp#631

0.15.0

Project

• BREAKING: Bumping minimal version of PHP to 7.3. Version is enforced by Composer and any older version is not allowed anymore. Please upgrade PHP accordingly.
• Docker image now uses PHP 7.3.
• Go images now use 1.15.

[Beam]

• Updated major portion of dependencies. Laravel was not updated yet.
• Added parameter article_id to AggregatePageviewLoadJob and AggregatePageviewTimespentJob commands.
• Fixed broken functionality of the segments flag is_article (available in pageview category). remp/remp#716
• Added health check (http://beam.remp.press/health) for database, Redis, storage and logging. remp/remp#735
• Added "Content Type" to article detail information.
• Fixed reseting paid_at column in conversions table on each row update. remp/remp#738

[Campaign]

• Fixed listing of banners over API (/api/banners).
• Added health check (http://campaign.remp.press/health) for database, Redis, storage and logging. remp/remp#735

[Mailer]

• BREAKING: Changed hermes restart implementation (handles restarts of hermes worker and mail worker). Please update your deploy process to set the deploy time to Redis or fallback to previous file-based restart (see config.local.neon.example for reference). remp/remp#736
  • Updated tomaj/hermes to version 3.0.0 which introduced RedisRestart implementation.
  • Switched default Fermes restart setting from file restart to redis restart.
  • File restart moved to config.local.neon.example as example configuration.
• Added parameter with_mail_types for GET /api/v1/mailers/templates endpoint, allowing to add details about mail_types assigned to templates. Documentation for the endpoint added. remp/crm#1450
• Added method to use Redis ping() command within MailCache. remp/remp#735
• Added health check (http://mailer.remp.press/health) for database, Redis, storage and logging. remp/remp#735
• Added graceful shutdown to MailWorkerCommand (worker:mail) via Tomaj\Hermes\Restart\RestartInterface. remp/remp#736

[Sso]

• Updated major portion of dependencies. Laravel was not updated yet.
• Added health check (http://sso.remp.press/health) for database, Redis, storage and logging. remp/remp#735

0.14.0

Project

• PHP CodeSniffer scripts (phpcbf, phpcs) updated to version 3.5.6 (now supporting PHP 7.3+).
• Added Gitlab CI test stage dependency on redis:3.2 version (tests now use real Redis instance).

Docker

• Fixed Elasticsearch index initialization for new installations.

[Beam]

• Added prevention of overlapping run of SnapshotArticlesViews command, which may have caused incorrect numbers in Beam dashboard concurrents graph.
• Added browser_id to Commerce model to expose it in commerce-related responses of Segments API.

[Campaign]

• Fixed change of missing campaign statistics caused by invalid pairing of data with labels due to inconsistent timezone use.
• Fixed possibility of zero campaign stats. Bug appeared if campaign included banner with an already removed variant with some stats tracked. remp/remp#628
• Fixed "how often to display" campaign rules, previously broken due to expiration of counter data in local storage. remp/remp#715
• Fixed possibility of zero campaign stats. Bug appeared if campaign included banner with an already removed variant with some stats tracked. remp/remp#628
• Removed redundant Tracker contract and related implementations. It was never used and necessary. Campaign should only consume Journal data, not produce them from backend.
• Increased campaigns backend stats fetch timeout to 5 seconds.

[Mailer]

• Fixed README.md typos, incorrectly linked classes, wording changes, small grammar fixes. remp/remp!390
• Upgraded nette/application to 2.4.16.
• Added support for click tracking configuration on mail template level. dn-mofa#50

[Sso]

• Added initial support for multiple providers. No real providers were actually added yet. #87

0.13.0

Important

Elasticsearch upgrade notice. In this version we are raising raising Elasticsearch compatibility to 7.. Up until now, Segments API supported Elasticsearch 6.. Please schedule the upgrade of database before proceeding to this version.

Release of newly build Tracker API and Segments API is required for application to work correctly.

We recommend two upgrade scenarios:

• When new release is ready, you can upgrade your existing cluster based on the documentation available at https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html.
• If you clear your Elastic data recurrently and archive stored events to CSV, you can spin up new v7 cluster and configure Telegraf to push data to both v6 and v7 of Elastic. Once you're satisfied with the amount of data in v7 (~1 month tends to be sufficient), wait for the next release and change Elastic address in Segments API to v7 cluster. If there are no issues, you can stop pushing new events to v6 cluster in Telegraf and stop the cluster completely.

[Beam]

• BREAKING: Application now requires Elasticsearch 7. remp/remp#616
  Please follow the upgrade steps:
  • Rebuild or download new Tracker and Segments binaries (binaries available at https://github.com/remp2020/remp/releases).
  • Omit type_name from Telegraf configuration (see Docker/telegraf/telegraf.conf docker configuration file for more details).
  • If you use default docker appliance to run REMP, please run:

    docker-compose stop beam_tracker beam_segments elasticsearch
    docker-compose build beam_tracker beam_segments elasticsearch
    docker-compose up -d beam_tracker beam_segments elasticsearch

• Go dep dependencies management system replaced with go modules. remp/remp#616
• Added ability to optionally specify (Elasticsearch) indexes prefix in .env for Tracker and Segments apps. remp/remp#616
• Added support for Elasticsearch authentication (auth parameter) in ElasticDataRetention and ElasticWriteAliasRollover commands. remp/remp#616
• Added content_type column to articles table. remp/remp#695
• Added optional parameter content_type in /api/v2/articles/upsert API endpoint. remp/remp#695
• Added optional parameter content_type in /api/articles/top API endpoint to filter articles by content_type. remp/remp#695
• Added optional parameter content_type in /api/authors/top API endpoint to filter articles by content_type. remp/remp#695
• Added optional parameter content_type in /api/tags/top API endpoint to filter articles by content_type. remp/remp#695

[Campaign]

• Fixed store pageview counts for campaign separately instead of globally. remp/remp#609
• Refactor campaign form pageview rules to use only every rule. remp/remp#609
• Added display N times and then stop rule to campaign form banner rules. remp/remp#609
• BREAKING Added migration to convert old campaign pageview rules to new format. May pause campaigns with not convertable rules. remp/remp#609
• Added new banner type – Overlay with Two Buttons and Signature. remp/remp#650

0.12.0

Important

Timespent data bug notice. There was a bug in aggregation of timespent data from Elastic to Beam's MySQL. If you have your timestamp data still online in Elastic, please run following command to reaggregate everything so you have correct data persisted:

# Alter the initial hour (2320) based on the amount of data you have in Elastic
# Calculate it as number of hours of stored data; or as HOURS(NOW() - EARLIEST_ELASTIC_DATA)

for i in {2320..0}; do sudo -u app-remp-beam php /data/web/remp2020.com/app/Beam/current/artisan pageviews:aggregate-timespent --now="-$i hours" --debug; done

If you don't have an online data and you don't have any other backup, consider the aggregated timespent data (in MySQL) aggregated before deploying this release as unreliable.

Please update the Segments API to fix the issue.

Sorry for an inconvenience.

Docker

• Added tzdata installation for remp_segments docker (required by golang).

Beam

• BREAKING: Changed way of specifying sections parameter in /api/articles/top API endpoint. Now sections can be filtered either using name or external_id parameters (before, name parameter was used implicitely). remp/remp#691
• Added ability to specify sections parameter in /api/authors/top and /api/tags/top/ API endpoints via name or external_id parameters (in the same fashion as in case of /api/articles/top). remp/remp#691
• Pageviews data for articles are now refreshed every minute instead of every hour. remp/remp#663
• Fixed ignored explicit browserId parameter in JS configuration. remp/remp#690
• Commands pageviews:aggregate-load and pageviews:aggregate-timespent do not show progress unless --debug parameter is specified.
• [Segments]: Fixed possibility of missing aggregations if Elastic was not able to resolve values for a sub aggregation because there were no records within the sub-aggregation branch.
• Fixed remplib.js generating undefined cookies when JS is run on a page with no query parameters. #81

Mailer

• Added handling for UserNotFoundException when confirming user in CRM. remp/remp#685
• Added notification on the settings screen about settings overridden by local config file. remp/remp#519

0.11.1

[Beam]

• Added environment variables to configure Redis databases in Laravel. remp/remp#671

[Campaign]

• Added environment variables to configure Redis databases in Laravel. remp/remp#671

[Sso]

• Added environment variables to configure Redis databases in Laravel. remp/remp#671

0.11.0

Docker

• BREAKING: Replaced spotify/kafka docker image with wurstmeister/kafka as original image was no longer maintained and new installations stopped working. remp/remp#638

  • In case you have existing installation in place using the docker compose, please run:

    docker-compose stop beam_tracker telegraf
    docker-compose rm beam_tracker telegraf
    docker-compose build beam_tracker telegraf

[Mailer]

• Added API endpoint mailers/mail-type-categories to list mail type categories. GH-64
• Extended API endpoint mailers/mail-types to include additional data and filter via public_listing and code parameters. GH-64
• Added API endpoint users/is-unsubscribed to check if user is explicitly unsubscribed from the newsletter. GH-64
• Added API endpoint users/logs-count-per-status to get number of emails sent to user per each status within provided timeframe. GH-64
• Added API endpoint users/logs to retrieve logs of emails sent to user. Various filter can apply, see README.md for more details. GH-64
• Added JSON schema validation to Subscribe and BulkSubscribe APIs. GH-64
• Added API endpoint users/preferences to read user's subscriptions to newsletters (mail types). GH-64
• Improved speed of job detail page - unsubscribe stats could slow rendering a bit if job was sent to 6+-figure recipients. remp/remp#624
• Added API endpoint mailers/mail-templates to list and filter available mail templates.
• Added early-version support for search in the top searchbox. Searchable are emails (templates), layouts, jobs and newsletter lists. GH-69
• Added early-version support for WYSIWYG editor in Template edit form. It's configurable in config.local.neon, Mailer keeps HTML editor as default for now. GH-58

[Beam]

• Added early-version support for search in the top searchbox. Searchable are articles, authors, sections, tags and segments. GH-62
• Added support for timezone parameter in Journal aggregations. remp/remp#605
• Quick range day filters now start from beginning of the day. remp/remp#605
• Added FORCE_HTTPS environment variable to enforce HTTPS generation to URLs instead of determining protocol based on the request. This is useful in case you're running your application on https, but internally use proxy forwarding the request via http. remp/remp#619
• Added new APIs api/authors/top and api/tags/top for retrieving top authors and tags per given time period. remp/web#366
• Articles upsert v2 api endpoint - process article titles only if they are present in payload. remp/remp#646
• Fixed remplib initialization which could use misidentification of user - remplib.getUserId() would return null even when the userId was correctly set in rempConfig. remp/remp#651

[Campaign]

• Added early-version support for search in the top searchbox. Searchable are banners and campaigns. GH-62
• Improved intervals in campaign stats charts. remp/remp#605
• Quick range day filters now start from beginning of the day. remp/remp#605
• Added FORCE_HTTPS environment variable to enforce HTTPS generation to URLs instead of determining protocol based on the request. This is useful in case you're running your application on https, but internally use proxy forwarding the request via http. remp/remp#619
• Fixed add new ab variant replaces last variant instead of adding new after last one. remp/remp#634
• Added option to disable banner events tracking. remp/remp#636
• Added ability to access banner properties in custom JS code run in banner via newly added params object. remp/remp#636
• Changed wording of hints in campaign's segment selection form. remp/remp#645
• Fixed remplib initialization which could use misidentification of user - remplib.getUserId() would return null even when the userId was correctly set in rempConfig. remp/remp#651

[Sso]

• Added FORCE_HTTPS environment variable to enforce HTTPS generation to URLs instead of determining protocol based on the request. This is useful in case you're running your application on https, but internally use proxy forwarding the request via http. remp/remp#619

0.10.0

Elasticsearch upgrade notice. We'll be raising Elasticsearch compatibility to 7.* in the first half of 2020 to keep with the latest changes. Current implementation Segments API is tested and maintained again Elasticsearch 6.*. Please plan your upgrade accordingly.

[Beam]

• Added support for conversion rate sorting in Conversions data table. remp/remp#306
• Fixed error thrown when using main search on authors listing. remp/remp#531
• Added command to maintain data retention for rolled-over Elastic indices. remp/remp#527
• Pageviews graph in article details page loads data from Journal snapshots instead of directly quering Journal API by default. Option PAGEVIEWS_DATA_SOURCE added to .env file to allow switching to old behaviour. remp/remp#442
• rempConfig.tracker configuration option explicit_referer_medium is deprecated and renamed to refererMedium. The old one is still accepted due to compatibility reasons.
• Added support for referer medium renaming - one can specify label for each medium by adding a record to referer_medium_labels table. remp/remp#543
• Updated layout footer with link to REMP website. remp/remp#522 HEAD
• Added ability to display external events in article detail and dashboard. remp/remp#574
• Added (optional) article_id field to event parameters pushed to tracker. remp/remp#556
• Fixed property naming validator issue checking unique constraint against account names, not property names.
• Added article conversion/pageviews stats filtering based on selected property. GH-50
• Added support for article tags (storing tags associated with articles, filtering in data tables). remp/remp#217
• Added top articles endpoint for listing top articles by time, sections. remp/web#1010
• Fixed duplicate conversions if multiple sections/tags/authors were linked to article.
• Fixed missing conversions if no section/tag/author was linked to article. remp/remp#586

[Campaign]

• Updated layout footer with link to REMP website. remp/remp#522
• Fixed missing validation rules for collapsible bar banner template. remp/remp#558
• Added support for campaign segment exclusion. User now need to match both inclusion and exclusion segment rules to see the campaign banner. GH-33
• Overal rectangle banner button text and main text attributes are now optional. This is useful when e.g. only using picture inside the banner. remp/remp#582
• Texts in collapsible bar, bar and medium rectangle bar templates are now optional. remp/remp#597

[Mailer]

• BREAKING: Attachment parameter in send-email API is now required to be base64-encoded to support PDF (and other binary) attachments.
• BREAKING: Context checking during email sending now only checks if user received an email with given context before; Mailer ignores mail_template_code being sent. In previous version, two different mail templates with same context could be sent to the same user. This version prevents such behavior. remp/crm#987
• BREAKING: Removed hardcoded support for Errbit/Airbrake error logging, added support for Sentry logging. See README for details on how to configure Sentry to track errors.
• Updated layout footer with link to REMP website. remp/remp#522
• Added possibility to filter hermes payload parameters in logs. Parameters password and token are already filtered by default. See config.neon for reference how to extend filtering with own parameters.
• Added hermes handler to unsubscribe users from mail type if emails are dropped. You can enable the feature in config.local.neon (see example file for reference). remp/remp#566
• Added configuration for allowed preflight (OPTIONS) headers into configuration. You can configure them via %preflight.headers parameter - see example use.
• Fixed possible duplicate email for same context being sent, in case the emails were scheduled at the same time via send-email API.
• Fixed and redesigned mailing list variants statistics to be sortable and filterable. remp/remp#593
• Fixed attachment_size column type - changing to integer (from incorrectly assigned datetime).
• Added option to select events to handle when starting Hermes worker in case there's need to run separate workers for mission-critical event types.

[Sso]

• Updated layout footer with link to REMP website. remp/remp#522