Outliers detection/removal component

[cvd-q]

Dear TFX community members,

I'm a student in Data Science from University of Padova (Italy) and I've decided to write my master's thesis on MLops. I'm very interested in TFX and I'd like to analyze it in depth. In particular, for the experimental part my supervisor and I have in mind a Beam/Spark (MapReduce) implementation of an outlier detection algorithm, especially to deal with large dataset, we think that such preprocessing step maybe helpful. Then we would like to contribuite to this project creating a custom component.

Could this idea be useful in some way? Are you planning to release data mining components?

Thanks a lot for your advice.

Contact

[email protected]

[rcrowe-google]

Could this idea be useful in some way? Are you planning to release data mining components?

Thanks Jiawei, and welcome to the group! Outlier detection is very useful in many domains. We have not really focused on data mining, but there is a lot of overlap between techniques for data mining and techniques for machine learning, especially in preprocessing. If you'd like to submit a proposal, please see the instructions in the proposals folder.

[cvd-q]

Thank you for replying and confirming that the idea is something interesting, as I thought in preprocessing. Now I'm going to discuss with the prof. about which algorithms are suitable to be implemented in Beam, I will write a proposal text as soon as a definitive decision will be made.