256 color terminal support

[mooskagh]

I suspect the answer is "no", but is there any chance to have 256 color support in the future?

Thanks!

[magiblot]

Yes! But not yet. I first need more free time, and then I'll have to address the following issues:

• Understand how 256/24-bit VT colors work and the logic and data structures needed (or maybe I could stop doing things myself and rely on a more sophisticated tool like notcurses).
• Design a public interface which:
  • Does not break backward compatibility.
  • Produces expected results whether the terminal supports more than 16 colors or not. In other words: the application developer should not need to care about terminal capabilities.
• Figure out if there's any chance of having hi-color palettes, which does not seem simple at a first glance. Otherwise, colors would have to be set manually in draw methods, like Turbo does, and they could not be used on predefined views like TDialog.

Cheers.

[pkral78]

I would like to point out these articles:
https://jexer.sourceforge.io/
and especially
https://jexer.sourceforge.io/evolution.html

[magiblot]

What do you expect to be able to do with 256/true color support?

[pkral78]

What do you expect to be able to do with 256/true color support?

Is this question about wider color support or moreabout general Sixel graphics?

[magiblot]

As I understand it, the OP's question is about text-mode color (https://gist.github.com/XVilka/8346728). I do not intend to support Sixel graphics.

[mooskagh]

Yeah I only meant text mode colors, and 256 colors would be more than enough for my case.

I was considering using tvision to implement a tool which would have a window with a very rich syntax highlighting, and it felt that 16 colors would not be enough (there were ~30 different categories and total, and also it would be nice to highlight similar categories by similar [but not completely the same] colors).
(in the end I decided to do graphical way and started to do that in GTK+)

Plus, 256 color text interfaces look much more beautiful, see for example this tool that is called GoAccess:

[electroly]

I had given some thought to how tvision on modern platforms might support extended colors in palettes, as I'm interested in that too. I'm willing to subclass all the controls and copy-paste their draw implementation from tvision so I can replace the color handling with some kind of custom mechanism, but if it could be weaseled into the existing palette support, so much the better. I'm specifically interested in allowing the user to pick custom dialog backgrounds, custom button colors, etc. Visual Basic for DOS allows this with a simple color picker and I thought it would be nice to have this in TMBASIC.

I noticed that blink doesn't work in a lot of modern terminals, and there's a bit dedicated to blink in palette color definitions. I don't think any of the built-in controls use blinking text (correct me if I'm wrong). Could a compile-time define allow that blink bit to be repurposed to mean "take the rest of the bits and use it as an index into an extended palette" where the extended palette has a similar structure, but with either 2-byte (256 color fg/bg) or 6-byte (24-bit) entries? With the compile-time symbol not defined, it would exclude that logic and the extended palette array and continue to do what tvision does now. Just a thought; I haven't looked into it very deeply, but I'm interested.

[electroly]

Oh, I have a second unrelated use case in TMBASIC. I want the user to be able to write TUI apps or full-screen console apps (e.g. colorful text-based games like ZZT), but critically I'd also like the ability to combine the two and have tvision dialogs floating above full-screen console games. Imagine like a betting window for a blackjack game that floats above a custom-drawn blackjack table. If tvision supports extended colors, I think I could run everything under tvision and simply draw onto the desktop for custom-drawn stuff, and then it's no big deal to occasionally pop dialogs up on top of it. In the absence of this support, I don't know how I'd pull it off if I wanted more than tvision's 8 bg and 16 fg colors; I think I'd have to make TUI and full-screen drawing mutually exclusive, using tvision for the former and using ncurses directly for the latter.

[magiblot]

Thank you everyone for the comments.

Integrating extended color support into the Turbo Vision API affects two different parts of the original API:

• The TDrawBuffer interface (moveStr, moveCStr, putAttribute...), which is pretty straight-forward to extend. This makes it possible to draw using extended colors without accessing TScreenCells directly. TScreenCell is already an API extension, as explained in the project's README.

• The Palette API. A TPalette contains an array of uchar. In views with an owner, the elements of this array are indices into the owner's array. Conversely, in views without an owner (e.g. the TApplication instance), the uchar elements must be interpreted as BIOS color attributes.

  So TPalette has the issue that it uses a single data type to represent two different things: indices and color attributes. The result is that when you define the palette of a class you cannot use the type system to express whether you are defining colors or indices.

  Extended color support is not incompatible with the fact that uchar is used for palette indices, but it does conflict with uchar being used for color attributes. A possible workaround to this is to give an special meaning to the blink bit (if I understood you well), so that instead of being a BIOS color attribute, it is an index into an array of extended attributes. I have not studied this possibility in depth yet, but I am worried that the Palette API will become more complex and so will the implementation of TColorSelector.

  But not all hope is lost. The Palette API is not only about the TPalette type, but also the uchar TView::mapColor(uchar color) method, which is responsible for translating a palette index into a color by traversing the view hierarchy. If we make this method virtual, it will be possible to bypass the TPalette lookup completely, which is something we would likely do anyway because the Palette API is already hard to use in itself (it takes a while to figure out the exact byte you have to modify to change the color of a view). But bypassing the Palette API implies that any palette-switching functionality will have to be implemented manually by the application. What do you think about this?

  By this point I have to admit that I have no idea how modern GUI toolkits manage the issue of color schemes. There's probably something we can learn from them.

I want the user to be able to write TUI apps or full-screen console apps (e.g. colorful text-based games like ZZT), but critically I'd also like the ability to combine the two and have tvision dialogs floating above full-screen console games.

I can confirm this will be possible at least by manipulating TScreenCell manually. This is notcurses-demo running inside a Turbo Vision application:

Cheers.

[electroly]

But not all hope is lost. The Palette API is not only about the TPalette type, but also the uchar TView::mapColor(uchar color) method ... bypassing the Palette API implies that any palette-switching functionality will have to be implemented manually by the application.

This sounds like a great approach; I like this. It's probably cleaner than hijacking the blink bit. Implementing my own mapColor for each control would be easy enough, and I could offer per-view palette customizations without having to override draw.

Would my mapColor implementations need to know whether the terminal supports extended colors? Seems like they would have to know and provide appropriate color mappings for the terminal's capabilities.

Very nice demo image. I'm excited about the possibilities!

[magiblot]

Would my mapColor implementations need to know whether the terminal supports extended colors? Seems like they would have to know and provide appropriate color mappings for the terminal's capabilities.

No. Turbo Vision will take care of converting the color you want to display to what the terminal can display. In other words: you should be able to use extended colors without knowing your terminal's capabilities or even knowing what a terminal is.

For example:

COLORTERM=truecolor (original)	TERM=xterm-256color

TERM=xterm-16color	TERM=xterm (bold as bright)

[magiblot]

But bypassing the Palette API implies that any palette-switching functionality will have to be implemented manually by the application.

This turns out to be too much of a disadvantage. Just think in how many classes we'd have to override the mapColor method to get the bottom panel right in this dialog:

It's not worth the effort. So I think it's a good idea to keep on using palettes instead of overriding mapColor. I found a way to accomplish this without increasing complexity:

[okbob]

st 24. 2. 2021 v 18:09 odesílatel magiblot <[email protected]> napsal:

But bypassing the Palette API implies that any palette-switching functionality will have to be implemented manually by the application. This turns out to be too much of a disadvantage. Just think in how many classes we'd have to override the mapColor method to get the bottom panel right in this dialog: [image: Screenshot_20210224_175029] <https://user-images.githubusercontent.com/20713561/109035732-397e1380-76c9-11eb-804c-18ba23c3ebcc.png> It's not worth the effort. So I think it's a good idea to keep on using palettes instead of overriding mapColor. I found a way to accomplish this without increasing complexity: [image: Screenshot_20210224_174047] <https://user-images.githubusercontent.com/20713561/109036204-a72a3f80-76c9-11eb-8906-55a27d246280.png> —

This looks pretty well You are receiving this because you are subscribed to this thread.

…

Reply to this email directly, view it on GitHub <#19 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEFO42Y526BGBFNFERMHK3TAUXFZANCNFSM4SU63LNA> .

[electroly]

FWIW, I'm probably already going to subclass all of the views that ship with tvision to add hooks for the BASIC bindings. It may be too much hassle for apps that just want to cook up a tvision GUI with fancy colors, but at least for TMBASIC's purposes I'm willing to do it since I'll do it once and they will serve all the BASIC programs the same way.

I didn't mention it above but the limit of 256 palette entries is kind of an issue on its own; unless I subclass and override draw or mapColor across the board, I won't be able to offer unlimited per-window palette customization for more than a couple windows because there aren't enough palette entries to go around:

// 0x08 - 0x0F: cpBlueWindow
// 0x10 - 0x17: cpCyanWindow
// 0x18 - 0x1F: cpGrayWindow
// 0x20 - 0x3F: cpGrayDialog
// 0x40 - 0x5F: cpBlueDialog
// 0x60 - 0x7F: cpCyanDialog

These six window styles already consume half of the available palette entries. I had to evict THelpWindow's palette (0x80 - 0x87) from my app palette because I needed to free up some palette entries.

In Visual Basic for DOS (and in modern Windows Forms) there is no concept of a palette, they just have a foreground and background color for each view that you can change individually. I wonder if an alternative solution here is to do something like that. What I really want is just the ability to set fields in, e.g., TButton that specify the button color, shadow color, unselected text color, and selected text color, with a default value (-1?) meaning "use the palette". If tvision offered these, there would be no need to subclass in order to set the colors. If I subclass the views and override either mapColor or draw to implement custom color handling, this is exactly how I'll present it to the callers.

[pkral78]

• So TPalette has the issue that it uses a single data type to represent two different things: indices and color attributes. The result is that when you define the palette of a class you cannot use the type system to express whether you are defining colors or indices.

So what about split this type into two? Is it feasible ?

[magiblot]

I've pushed extended color support into the colors branch. I'd like to make sure it doesn't break anything in @electroly's project (the only active one I know of) before merging it into master.

There's a new section at the end of the README that explains briefly how it works.

Basically, it comes down to the following:

• There is a new type named TColorAttr which replaces uchar. It specifies a foreground and background colors and a style. Colors can be specified as BIOS color attributes, 24-bit RGB values or xterm-256color palette indices. Styles are the typical ones (bold, italic, underline...).
• The TDrawBuffer methods, which used to take uchar or ushort parameters to specify color attributes, now take TColorAttr.
• TPalette, which used to contain an array of uchar, now contains an array of TColorAttr. TView::mapColor also returns TColorAttr.
• TView::mapColor has been made virtual so that you can bypass the palette system without having to rewrite any draw methods.

Turbo Vision takes care of mapping the provided colors to the current terminal capabilities. Client applications need not worry about this.

Speaking of terminal capabilities: for the moment, these are only detected properly on Unix systems (where Ncurses/Termcap/Terminfo are widely used). Windows applications are capped to 16 colors because I have yet to investigate what can be done there (although setting the COLORTERM environment variable to truecolor or 24bit will unlock true color support in either case).

Unfortunately, I haven't yet published any application that allows you to play with these features, so you'll either have to wait or attempt to write one yourselves.

Cheers.

[electroly]

I updated to tvision commit 79182b1 and ran a build and test on all platforms, no problems at all. Looks good! I'll give the new capabilities a try.

[electroly]

I whipped up a quick 256-color test. Looks good! I tested both overriding mapColor and passing TColorAttr to TDrawBuffer. Nice work!

[magiblot]

Great!