Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SuperEditor and SuperReader layer system to provide app-level composition (iOS)(Relates to #424, #893, #1166)(Resolves #1508) #1470

Merged
merged 37 commits into from
Oct 23, 2023

Conversation

matthew-carroll
Copy link
Contributor

@matthew-carroll matthew-carroll commented Sep 26, 2023

This PR includes changes to a number of different areas, which are all somewhat related to working with document layers in SuperEditor and SuperReader. We need to ensure that our layer system makes it possible to achieve obvious goals. This PR stress tests our layers by reworking our iOS caret, handles, magnifier, toolbar, and floating cursor display. This PR also includes some changes to the content layer system to achieve that goal.

Proving Layer Capabilities by Adjusting iOS Editing Controls

Super Editor displays a number of controls on iOS: caret, drag handles, magnifier, toolbar, and a floating cursor. All of these controls were primarily managed by IosDocumentTouchInteractor. This interactor was constantly scheduling frames to inspect the document layout and figure out where to place the various controls. This coordination was complicated, convoluted, fragile, and non-performant.

To prove the viability of our layer system, I aimed to do a few things:

  1. Move caret and handles into a document layer. They're based on document layer, so they make sense as a document layer. Also, we have a ticket that wants them beneath the onscreen keyboard, which happens automatically with this change - [SuperEditor] Carat, selection handles, and overlays should hide behind above-keyboard toolbars #893
  2. Make caret and drag handles configurable via widget composition instead of creating special SuperEditor properties for that purpose.
  3. Make the toolbar and magnifier configurable via widget composition instead of creating special SuperEditor properties to build it.

When breaking apart the iOS controls to be more composable, it became clear that relying on SuperEditor properties for all such controls was getting out of hand. We'd need to also expose similar controls for Android, and possibly other platforms. Therefore, I shifted the approach for shared control. I created a widget called SuperEditorIosControlsScope, which provides access to a SuperEditorIosControlsController, which holds the status of the magnifier, the toolbar, and the floating cursor. SuperEditor internally adds a SuperEditorIosControlsScope above the document layers in the widget tree, so that all document layers have access to a shared context. Also, an app can surround its SuperEditor with its own SuperEditorIosControlsScope to further share visibility into those controls. This allows apps, for example, to build their own toolbar widget.

New Floating Cursor Architecture

The previous floating cursor behavior was spread through widgets in a confusing way. This was primarily due to two factors. First, the floating cursor changes originate within the IME system, which is otherwise dedicated to text editing. So it's a weird starting point (but not in our control). Second, our iOS touch interactor was calculating the visual size and offset for the floating cursor, as well as updating the document selection when it moved. But the floating cursor had nothing to do with Flutter gestures, so this was a weird widget to handle those responsibilities.

As of this PR, the responsibility breakdown is as follows:

SuperEditor
  SuperEditorIosControlsScope (owns the FloatingCursorController, available to all descendent widgets)
    SuperEditorImeInteractor (reports floating cursor start/move/end from IME)
      EditorFloatingCursor (decides where the floating cursor should appear, paints it, and updates the doc selection)
        IosDocumentTouchInteractor (when floating cursor is active - auto-scrolls up/down)
          IosHandlesDocumentLayer (when floating cursor is active - hides caret/makes it gray, stops it from blinking)

Ideally, the auto-scrolling behavior in IosDocumentTouchInteractor should probably be handled by EditorFloatingCursor. However, I think this change would require that we put the concept of SuperEditor scrolling in its own widget, so that all descendent widgets have an opportunity to auto-scroll, or to change the scroll offset. I filed a ticket for this: #1495

Composable toolbar and magnifier

The iOS toolbar and magnifier are now configurable.

Example of overriding the default iOS toolbar and magnifier:

  @override
  void initState() {
    super.initState();
    
    _iosControlsController = SuperEditorIosControlsController(
      toolbarBuilder: _buildIosFloatingToolbar,
      magnifierBuilder: _buildIosMagnifier,
    );
  }

  @override
  void dispose() {
    _iosControlsController.dispose();

    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return SuperEditorIosControlsScope(
      controlsContext: _iosControlsController,
      child: SuperEditor(
         // ...
      ),
    );
  }


  Widget _buildIosToolbar(BuildContext context) {
    return IOSTextEditingFloatingToolbar(
      onCutPressed: _cut,
      onCopyPressed: _copy,
      onPastePressed: _paste,
      focalPoint: _selectionLayerLinks.expandedSelectionBoundsLink,
    );
  }

  Widget _buildIosMagnifier(BuildContext context, LeaderLink focalPoint) {
    return Center(
      child: IOSFollowingMagnifier.roundedRectangle(
        leaderLink: focalPoint,
        offsetFromFocalPoint: const Offset(0, -72),
      ),
    );
  }

Editors vs Reader Controls Scope

Editors have some controls that readers don't have, e.g., caret and floating cursor. Therefore, SuperEditor and SuperReader should use different types of controllers, and therefore different types of scopes for overlay controls.

Super Editor has:

SuperEditorIosControlsScope - widget that owns a controller and can be used to customized the controls.

SuperEditorIosControlsController - owns the state of overlay controls: handle color, floating cursor, toolbar builder, etc.

SuperEditorIosToolbarOverlayManager - widget inside of SuperEditor that adds and removes the toolbar from the app overlay.

SuperEditorIosMagnifierOverlayManager - widget inside of SuperEditor that adds and removes the magnifier from the app overlay.

SuperEditorIosToolbarFocalPointDocumentLayerBuilder - document layer widget that places a rectangle around the selection as a focal point for the toolbar overlay widget.

SuperEditorIosHandlesDocumentLayerBuilder - document layer widget that displays the caret and drag handles

Super Reader has:

SuperReaderIosControlsScope - same as above but for SuperReader

SuperReaderIosControlsController

SuperReaderIosToolbarOverlayManager

SuperReaderIosMagnifierOverlayManager

SuperReaderIosToolbarFocalPointDocumentLayerBuilder

SuperReaderIosHandlesDocumentLayerBuilder

Additions and Changes to Help with Editor/Reader Document Layers

In theory, all apps could use existing content layer widgets, like ContentLayerStatelessWidget and ContentLayerStatefulWidget to implement document layers. However, all such layers would need to obtain a reference to the content StatefulElement, obtain its State, and then type cast it to a DocumentLayout. This PR introduces
DocumentLayoutLayerStatelessWidget and DocumentLayoutLayerStatefulWidget, which do this work on the app's behalf and make it easy to quickly position content based on a DocumentLayout.

I utilized the new DocumentLayoutLayerStatefulWidget to simplify the existing SelectionLeadersDocumentLayer implementation.

Content Layer Changes

During work on this ticket, I discovered that my StatefulWidget layers were constantly re-inflating their widget on every frame. This rendered the State object useless. I altered ContentLayersElement to use forgetChild() on the layers instead of deactivateChild(), which allows ContentLayersElement to avoid building those widgets, while also retaining the associated Element and State of each layer so they can be re-added to the tree later in the frame.

A ContentLayerState can query its cached layout information, instead of being forced to only receive it in doBuild(). This allows for event callbacks, such as user interactions, to operate on that same cached layout data.

ContentLayerState used to provide its associated RenderObject to computeLayoutData() but now it provides both its Element and RenderObject. This change is used in this PR, for example, to access the State of a StatefulElement and then interact with that State as a DocumentLayout.

Changes to text selection bounds calculations

Previously, calculating a selection rectangle at a collapsed selection (e.g., at a caret location) returned a zero-sized rectangle. In general, it's more useful to return a zero-width rectangle that's as tall as the nearest character. I changed the calculation to return the latter.

The above statement is no longer true, because I copied that position rectangle behavior into a long-press selection PR that's already been merged. So the behavior exists, but this PR isn't the first place that it was added.

Minor Tasks in this PR:

  • Added padding to Example App floating action buttons to keep them above mobile keyboard controls.
  • Migrated iOS magnifier placement from Flutter's LayerLink to follow_the_leader LeaderLink.
  • Adjusted control character test so that it correctly ensures focus doesn't move. As a result I removed the test for TAB because TAB probably should move focus by default. We may be missing TAB behavior, but we can add that later.
  • Added tests to ensure that toolbar and magnifier are hidden on web for iOS. That original work was done in [SuperEditor][web] Defer to browser toolbar and magnifier (Resolves #1390) #1483 but no tests were added and this PR completely changed the toolbar and magnifier code, so we needed tests to ensure it wasn't broken by this PR.

…ystem (Resolves #424)

 - Applied floating cursor expanded selection fix from another PR. Fixed floating cursor auto-scroll bottom boundary position bug. Also debugging a floating cursor document
   mapping error when moving between nodes.
 - iOS controls configurability from outside Super Editor
 -  Moving complexity out of IosDocumentTouchInteractor. Currently trying to remove all possible dependencies on the shared and centralized IosDocumentGestureEditingController. All tests pass.
 - Removed a bunch of unneeded properties from IosDocumentTouchInteractor. All tests pass.
 - Refactored SuperReader to use new layer controls approach. All tests pass.
@matthew-carroll matthew-carroll changed the title Update SuperEditor and SuperReader layer system to provide app-level composition (iOS)(Relates to #424, #1166) Update SuperEditor and SuperReader layer system to provide app-level composition (iOS)(Relates to #424, #893, #1166) Sep 26, 2023
… follow_the_leader updated to fix arrow alignment. All tests pass.
 - merged in long-press selection for iOS and Android
… to eventually migrate everything to LeaderLinks.
…kenly using the Interactors context. Changed iOS toolbar overlay manager to require the focal point because the Overlay can't ever be a descendent of the iOS controls scope.
…ixed an issue where collapsed selections were reported as expanded because text positions for the same offset with different affinities reported as unequal.
…preparation to cleanup how the floating cursor is managed across various widgets and classes.
…ow the magnifier builder works - also added default toolbar builders to SuperEditor and SuperReader, which are used when the toolbar builder is null.
 - recovered iOS controls hidden on web for SuperEditor
 - added inspector for overlay controls in SuperEditor
 - added tests for hidden controls on web for SuperEditor
…andlesDocumentLayerBuilder

Renamed IosControlsDocumentLayer to IosHandlesDocumentLayer
@matthew-carroll matthew-carroll marked this pull request as ready for review October 13, 2023 19:45
@matthew-carroll
Copy link
Contributor Author

@angelosilvestre this is a really big PR. If you want to review it in an unusual way, let me know. We can review by behavior, or by small groups of files. We can also jump on a call at some point if you think that's necessary for the review.

I'd like to get this merged ASAP, but I also know that this review is gonna be a lot of work.

@matthew-carroll
Copy link
Contributor Author

matthew-carroll commented Oct 13, 2023

CC @miguelcmedeiros @brian-superlist @Jethro87 @jmatth - this PR changes the architecture for how SuperEditor and SuperReader configure and position the caret, handles, floating cursor, magnifier, and toolbar. Related to those changes, this PR also makes changes to the existing gesture interactor, because I moved all the non-gesture behavior out of there into other places, as mentioned in this PR description.

@matthew-carroll matthew-carroll changed the title Update SuperEditor and SuperReader layer system to provide app-level composition (iOS)(Relates to #424, #893, #1166) Update SuperEditor and SuperReader layer system to provide app-level composition (iOS)(Relates to #424, #893, #1166)(Resolves #1508) Oct 14, 2023
Copy link
Collaborator

@angelosilvestre angelosilvestre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't look through all of the code yet, but I left some initial comments.

_controlsContext!.floatingCursorController.cursorGeometryInViewport
.removeListener(_onFloatingCursorGeometryChange);
}
_controlsContext = SuperEditorIosControlsScope.rootOf(context);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be looking for the nearest InheritedWidget? I assume SuperEditor is always adding its own SuperEditorIosControlsScope in the widget tree. If that's the case, it could add the SuperEditorIosControlsScope only if there isn't an ancestor one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we'd use the nearest because that's more performant. You've described a solution for the case of two scopes, one inside SuperEditor and one outside SuperEditor. But what about when there's 3, 4, or 5 scopes? Which one do we use? The nearest or the root? I think the assumption of such a scope is that the outermost scope is the one that takes priority...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually, when working with InheritedWidgets I think the assumption is that the nearest is always used...

But what about when there's 3, 4, or 5 scopes? Which one do we use?

My guess is that we should always use the nearest. If we always look for the outermost scope, then it doesn't matter if we have 2 scopes or 10.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But how does that policy facilitate our users? Imagine that Package includes a SuperEditor and wraps it with an iOS controls scope to customize the controls. Then App uses Package which uses SuperEditor. Now the app wants to further change those controls by wrapping with another iOS controls scope. That won't have any effect if we look for the nearest ancestor.

What would App do to customize the controls within Package, which customized the controls within SuperEditor?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understood, with the given example it makes sense to pick the outermost scope.

…d SuperReader and deprecated them to avoid unexpected compilation issues
_controlsContext!.floatingCursorController.cursorGeometryInViewport
.removeListener(_onFloatingCursorGeometryChange);
}
_controlsContext = SuperEditorIosControlsScope.rootOf(context);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually, when working with InheritedWidgets I think the assumption is that the nearest is always used...

But what about when there's 3, 4, or 5 scopes? Which one do we use?

My guess is that we should always use the nearest. If we always look for the outermost scope, then it doesn't matter if we have 2 scopes or 10.

super_editor/lib/src/default_editor/super_editor.dart Outdated Show resolved Hide resolved
super_editor/lib/src/default_editor/super_editor.dart Outdated Show resolved Hide resolved
super_editor/lib/src/infrastructure/blinking_caret.dart Outdated Show resolved Hide resolved
 * Brought back iOS SuperTextField toolbar focus arrow, but now using Leader/Follower instead of direct offsets
Copy link
Collaborator

@angelosilvestre angelosilvestre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matthew-carroll
Copy link
Contributor Author

@angelosilvestre do you feel like you have a good idea about what this PR is trying to achieve and how it does it? I want to make sure that you really feel good about what's here before merging and moving forward with the other platforms.

@matthew-carroll
Copy link
Contributor Author

I'm gonna go ahead and merge this to move on to Android. @angelosilvestre please let me know if you have any architectural concerns about this approach.

@matthew-carroll matthew-carroll merged commit b2abadb into main Oct 23, 2023
10 of 11 checks passed
@matthew-carroll matthew-carroll deleted the 424_create-generic-content-anchor-system branch October 23, 2023 17:51
github-actions bot pushed a commit that referenced this pull request Oct 23, 2023
matthew-carroll added a commit that referenced this pull request Oct 23, 2023
dxvid-pts pushed a commit to dxvid-pts/super_editor that referenced this pull request Feb 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants