Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable substitutions #1471

Merged
merged 77 commits into from
Sep 6, 2024
Merged
Changes from 1 commit
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
510c2d9
Start on pcg builder
lockshaw Jun 4, 2024
7b55ed1
Add tests and some implementation for pcg builder
lockshaw Jun 4, 2024
c379efd
Add pcg tests, make dtgen constructors explicit to fix bug
lockshaw Jun 10, 2024
35fa653
Add remainder of PCG tests
lockshaw Jun 10, 2024
865a28e
Merge remote-tracking branch 'origin/repo-refactor' into pcg-builder
lockshaw Jun 10, 2024
f379539
Fix build issues in local-execution
lockshaw Jun 10, 2024
2dbb3b9
Format
lockshaw Jun 10, 2024
4050c99
Address Reyna comments, add topological_order function for PCG
lockshaw Jun 17, 2024
42c1968
Pre multidigraph refactor
lockshaw Jun 19, 2024
3be816f
Removing visitable from sp code
lockshaw Jun 21, 2024
6d68324
Add open dataflow graph, start to replace pcg dataflow graph
lockshaw Jun 23, 2024
64a3403
Start refactoring substitutions
lockshaw Jun 24, 2024
7d4c7be
Add utility functions to support pattern matching
lockshaw Jun 25, 2024
9ab9eb2
Pre-refactor inputs
lockshaw Jun 26, 2024
7ae7c65
Merge remote-tracking branch 'origin/repo-refactor' into dataflow-graph
lockshaw Jun 26, 2024
f9b129e
Fix proj url
lockshaw Jun 26, 2024
cf73f08
Get back to substitutions, now with unordered graph inputs
lockshaw Jul 7, 2024
5fd666d
Get substitutions building
lockshaw Jul 13, 2024
5f0c88a
substitutions-tests now builds
lockshaw Jul 13, 2024
3228f2d
Fix bug in filter, pass some initial substitution tests
lockshaw Jul 14, 2024
5f4cc01
Add tests for fmt::to_string, fix some substitutions bugs
lockshaw Jul 15, 2024
ad60be0
Pass initial unit tests for find_pattern_matches
lockshaw Jul 15, 2024
a972da2
Start on unit tests for pcg pattern
lockshaw Jul 15, 2024
bcf776e
Pass initial test for find_pattern_matches
lockshaw Jul 19, 2024
e28400e
Merge remote-tracking branch 'origin/repo-refactor' into dataflow-graph
lockshaw Jul 19, 2024
fe6d65d
Fix small build issue in tests
lockshaw Jul 19, 2024
e647af7
Format
lockshaw Jul 19, 2024
8b58760
Sync tests in CI with tests in proj
lockshaw Jul 19, 2024
1fafb9d
Fix minor build errors in kernels and local-execution
lockshaw Jul 19, 2024
0804314
Format
lockshaw Jul 19, 2024
dd5465c
Remove outdated code
lockshaw Jul 20, 2024
29ec5b8
More outdated code removal
lockshaw Jul 20, 2024
ff41743
More cleanup, add test for sp decomposition
lockshaw Jul 20, 2024
e71d200
Pull apart containers.h
lockshaw Jul 21, 2024
c06710c
More sp testing and fixes
lockshaw Jul 21, 2024
2f75566
Break up graph algorithms.h
lockshaw Jul 21, 2024
c81d3a4
Pre- full SP algo commit
lockshaw Jul 23, 2024
2a11c7e
Add initial implementation and tests for cbc decomposition and invers…
lockshaw Jul 23, 2024
71a9e0f
Pass test for get_inverse_line_graph
lockshaw Jul 24, 2024
25eb1db
Add new multidigraph
lockshaw Jul 24, 2024
64f1932
Fix get_inverse_line_graph to return a MultiDiGraph instead of a DiGraph
lockshaw Jul 24, 2024
31c8d17
Add tests for parallel and series reduction finding
lockshaw Jul 24, 2024
19e7e28
Add really rough implementation of valdez sp decomposition
lockshaw Jul 24, 2024
3791e86
Fix local-execution build
lockshaw Jul 25, 2024
267b72d
Add implementations and tests for applying series/parallel reductions
lockshaw Jul 25, 2024
bb2769a
Format
lockshaw Jul 26, 2024
39cb7b3
Clean up sp decomposition interface and tests
lockshaw Jul 27, 2024
ce0234d
Format
lockshaw Jul 27, 2024
3dc3ec6
Add comments for top-level substitutions functions, add proj doxygen …
lockshaw Jul 31, 2024
ee518c2
Start sketching out substitutions code
lockshaw Jul 31, 2024
f69b95a
Merge branch 'dataflow-graph' into substitutions-fix
lockshaw Jul 31, 2024
3c06b88
Fix build errors
lockshaw Aug 1, 2024
3d6f681
Add ability to permute node ids
lockshaw Aug 1, 2024
098a9d1
Cleanup and start to test new substitutions code
lockshaw Aug 4, 2024
9bd4f14
Add test case for evaluate_substitution_output
lockshaw Aug 5, 2024
101083b
Add naive isomorphism detection code
lockshaw Aug 5, 2024
9fec50c
Add graph inputs to open dataflow graph isomorphism
lockshaw Aug 6, 2024
7c60736
Add input permutation to evaluate_substitution_output
lockshaw Aug 6, 2024
cb6eab2
Fix permute_node_ids
lockshaw Aug 8, 2024
2f3d67a
Add test for permute_input_ids
lockshaw Aug 8, 2024
03cbd02
Migrate over to mutable implementation of apply_substitution
lockshaw Aug 23, 2024
4a8deae
Add fast isomorphism checking and an initial implementation of full s…
lockshaw Aug 24, 2024
0757e94
Pass initial full substitutions test
lockshaw Aug 24, 2024
ba0a174
Cleanup old isomorphism checking code
lockshaw Aug 24, 2024
4dfa403
Merge remote-tracking branch 'origin/repo-refactor' into substitution…
lockshaw Aug 24, 2024
f156f96
Fix post-merge bugs
lockshaw Aug 24, 2024
5f09298
Fix broken pcg builder test
lockshaw Aug 26, 2024
deff4f8
Format
lockshaw Aug 26, 2024
d71d24f
Reorganize code and remove some outdated code pre-code-review
lockshaw Aug 26, 2024
1a63f90
Format
lockshaw Aug 26, 2024
aecbbe6
Address review comments
lockshaw Aug 31, 2024
afc6f7f
Address missed comment
lockshaw Aug 31, 2024
7d6fadf
Remove latex dependency to avoid CI out-of-disk-space
lockshaw Aug 31, 2024
f804308
Format
lockshaw Aug 31, 2024
82c42d7
Merge remote-tracking branch 'origin/repo-refactor' into substitution…
lockshaw Sep 4, 2024
8499ed1
Fix build issues
lockshaw Sep 5, 2024
9e39e08
Fix incorrect test case
lockshaw Sep 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Pass initial unit tests for find_pattern_matches
lockshaw committed Jul 15, 2024
commit ad60be0ce09a22bd6b79ccd877b3d2b967848a66
Original file line number Diff line number Diff line change
@@ -88,7 +88,7 @@ std::vector<UnlabelledDataflowGraphPatternMatch>
merge_unlabelled_dataflow_graph_pattern_matches(prefix_match,
postfix_match,
subpatterns.subpattern_1_outputs_to_subpattern_2_inputs);
if (unsplit.has_value()) {
if (unsplit.has_value() && unlabelled_pattern_does_match(pattern, graph, unsplit.value(), additional_criterion)) {
matches.push_back(unsplit.value());
}
}
9 changes: 0 additions & 9 deletions lib/substitutions/test/src/test_pattern_matches.cc
Original file line number Diff line number Diff line change
@@ -93,7 +93,6 @@ TEST_SUITE(FF_TEST_SUITE) {
NodeAddedResult n0_added = graph.add_node({}, 1);
Node n0 = n0_added.node;
OpenDataflowValue v0 = OpenDataflowValue{get_only(n0_added.outputs)};
// CHECK(v0 == OpenDataflowValue{DataflowOutput{n0, 0}});

NodeAddedResult n1_added = graph.add_node({v0}, 1);
Node n1 = n1_added.node;
@@ -223,13 +222,5 @@ TEST_SUITE(FF_TEST_SUITE) {

CHECK(matches == correct);
}

//
// for (UnlabelledDataflowGraphPatternMatch const &match : matches) {
// CHECK(unlabelled_pattern_does_match(pattern,
// graph,
// match,
// match_additional_crition_always_true()));
// }
}
}
16 changes: 0 additions & 16 deletions lib/utils/include/utils/optional.decl

This file was deleted.

28 changes: 2 additions & 26 deletions lib/utils/include/utils/optional.h
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
#ifndef _FLEXFLOW_UTILS_INCLUDE_UTILS_OPTIONAL_H
#define _FLEXFLOW_UTILS_INCLUDE_UTILS_OPTIONAL_H

#include "fmt.h"
#include "rapidcheck.h"
#include <rapidcheck.h>
#include "utils/exception.h"
#include "utils/optional.decl"
#include "utils/fmt/optional.h"

namespace FlexFlow {

@@ -38,29 +37,6 @@ std::optional<std::invoke_result_t<F, T>> transform(std::optional<T> const &o,

} // namespace FlexFlow

namespace fmt {

template <typename T, typename Char>
struct formatter<
::std::optional<T>,
Char,
std::enable_if_t<!detail::has_format_as<std::optional<T>>::value>>
: formatter<std::string> {
template <typename FormatContext>
auto format(::std::optional<T> const &q, FormatContext &ctx)
-> decltype(ctx.out()) {
std::string result;
if (q.has_value()) {
result = fmt::to_string(q.value());
} else {
result = "nullopt";
}
return formatter<std::string>::format(result, ctx);
}
};

} // namespace fmt

namespace rc {

template <typename T>